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ABSTRACT 


The  MK  92  Mod  2  Fire  Control  System  (FCS)  is  a  complex,  maintenance  intensive 
shipboard  weapon  system  found  primarily  aboard  the  Oliver  Hazard  Perry  class  guided  missile 
frigates  (FFG-7).  This  system,  based  on  1970's  technology,  frequently  requires  extensive 
troubleshooting  and  supplemental  support  from  shore-based  technical  experts.  A  maintenance 
advisor  expert  ^stem  (MAES)  is  bong  developed  by  the  Port  Hueneme  Division  of  the  Naval 
Surface  Warfare  Center  (NSWC-PHD)  and  the  Naval  Postgraduate  School  (NPS)  to  assist 
the  Fire  Control  technicians  aboard  ship  to  better  isolate  faults  in  the  MK  92  Mod  2  FCS. 

This  thesis  furthers  the  efforts  of  the  project  at  NPS  by  investigating  key 
implementation  issues  that  will  affect  the  deployment  of  the  initial  version  of  MAES  to  the 
fleet.  Additional  deployment  issues  addressed  in  the  thesis  include  incorporating  lessons 
learned  from  deploying  other  ejq)ert  systems  in  the  armed  forces,  gaining  support  from 
individual  chains  of  command,  training  MAES  users  effectively,  involving  MAES  users  in  the 
implementation  process,  and  changing  hardware  implementation  issues.  A  training  plan, 
implementation  plan,  and  updated  MAES  user’s  manual  for  the  initial  deployment  are 


included. 
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I.  INTRODUCTION 


A  developmait  project  initialized  by  the  Naval  Surface  Warfare  Cento*,  Port  Hueneme 
4C00  (NSWC-PHD),  supported  by  the  Naval  Postgraduate  School  (NFS),  and  sponsored  by 
the  Naval  Sea  Systems  Command  (NAVSEA),  the  Mark  92  (MK  92)  Mod  2  Fire  Control 
System  (FCS)  Maintenance  Advisor  Expert  System  (MAES)  project  was  initiated  in  1992. 
It  will  serve  as  a  tool  to  aid  shipboard  Fire  Control  technicians  in  diagnosing  and 
troubleshooting  faults  that  occur  in  the  MK  92  FCS.  MAES  has  other  benefits  that  include: 

•  Reduced  mean-time-to-repair  (MTTR)  which  results  in  unproved  operational 
readiness. 

•  Significant  cost  savings  from  a  reduction  in  the  number  of  No  Fault  Evident  (NFE) 
circuit  cards  replaced. 

•  Reduced  reliance  on  outside  technical  support  activities. 

•  A  capability  to  more  efifectively  train  junior  fire  control  technicians  in  the 
troubleshooting  of  difificult  system  faults  in  general. 

•  Proving  the  value  of  expert  systems  which  may  lead  to  the  development  of  expert 
systems  for  other  complex  systems  such  as  the  MK86  GFCS  or  the  SQQ-28 
SONAR  processor. 

Initial  prototype  versions  of  MAES  have  been  developed  and  tested.  Copies  of  the  initial 
version  have  been  distributed  for  evaluation  to:  Fleet  Training  Center,  San  Diego  (FTC-SD), 
USS  Sides  (FFG-14),  USS  Wadsworth  (FFG-19),  and  Fleet  Combat  Training  Center  (FCTC), 
Norfolk.  In  Jialy  1995,  under  the  sponsorship  of  Commander  Naval  Surface  Forces  Atlantic 
(COMNAVSURFLANT),  a  Navy  Sdentific  Assistance  Program  (NSAP)  grant  was  approved 
that  would  support  the  implementation  and  evaluation  of  the  system  aboard 
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COMNAVSURFLANT  sJiips.  The  next  step  will  be  the  deployment  of  the  initial  production 
version  onboard  six  East  coast  based  frigates.  An  effective  implementation  plan  is  essential 
in  the  transition  of  the  system  from  a  prototype  to  a  deployable  system  and  is  the  focus  of  this 
thesis. 

A.  OBJECTIVES 

This  research  is  directed  toward  the  successfiil  implementation  of  the  initial 
deployment  version  of  MAES.  Proper  management  of  the  deployment  process  is  critical  to 
the  effective  allocation  of  resources.  The  thesis  has  three  main  objectives.  The  first  is  to 
determine  the  potential  implementation  issues  that  may  affect  the  initial  deployment  of  MAES. 
The  second  is  to  apply  the  applicable  implementation  issues  to  the  deployment  of  MAES. 
The  third  objective  is  the  development  of  an  implementation  plan,  training  plan,  and  a  revised 
MAES  user’s  manual  that  address  the  implementation  issues. 

B.  RESEARCH  QUESTIONS 

This  thesis  attempts  to  answer  the  following  research  questions: 

1.  Primary  Research  Question 

•  What  are  the  potential  implementation  issues  for  the  initial  fielding  of  MAES? 

2.  Subsidiary  Research  Questions 

•  How  can  the  identified  implementation  issues  be  addressed  for  the  deployment  of 
the  initial  version  of  MAES? 

•  What  are  the  end-user  training  concepts  to  be  applied  to  the  formulation  of  a 
MAES  training  plan  for  shipboard  technicians? 
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•  What  are  the  hardware  implementation  issues  associated  with  the  initial 
deployment  of  MAES? 

•  What  are  the  lessons  learned  from  the  implementation  of  other  expert  systems  for 
weapons  ^sterns  within  DOD  and  to  what  degree  are  they  applicable  to  the 
MAES  deployment? 

C.  SCOPE 

The  scope  of  the  theas  is  limited  to  examining  the  initial  deployment  implementation 
issues  and  development  of:  (1)  An  implementation  plan  for  the  initial  deployment  of  MAES; 
(2)  A  training  plan  for  shipbo^d  technicians  to  be  used  during  the  initial  deployment;  and  (3) 
An  updated  User’s  Manual. 

D.  METHODOLOGY 

The  research  methodology  for  this  thesis  includes  a  thorough  literature  review  of 
software  applications  implementation  with  an  emphasis  on  expert  system  implementation.  In 
addition,  interviews  with  personnel  from  other  DOD  activities  that  have  implemented  expert 
syst«ns  were  conducted.  Addressing  the  implementation  issues  for  the  actual  deployment  of 
MAES  follows  the  implementation  framework  developed  by  Prerau  (1990). 

E.  THESIS  ORGANIZATION 

This  thesis  consists  of  four  chapters  and  five  appendices.  The  following  is  a  brief 
description  of  the  contents  of  each  chapter. 

Chapter  n  provides  the  reader  with  the  issues  associated  with  implementing  a  software 
application.  The  development  model  developed  by  Prerau  (1990)  is  presented  and  is  being 
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used  to  implement  MAES.  The  Lucas-Ginzberg  (1990)  model  for  implementation  is 
introduced  to  determine  the  implementation  fectors  which  need  to  be  considered.  Differences 
in  expert  system  implementation  are  pointed  out. 

Chapter  m  establishes  the  implementation  issues  that  can  be  influenced  by  the 
deployment  team.  Different  methods  of  influencing  implementation  issues  are  explored. 

Chapter  IV  evaluates  the  implementation  issues  determined  in  Chapter  n  and  applies 
them  to  the  implementation  of  the  initial  version  of  MAES.  Lessons  learned  fi'om  the 
deployment  of  other  DOD  diagnostic  expert  systems  are  discussed. 

Chapter  V  provides  a  thorough  review  of  the  duties  and  responsibilities  of  tiie 
deployment  team.  Hardware  implementation  issues  are  discussed  as  they  apply  to  the 
deployment  of  MAES.  The  chapter  also  discusses  the  contents  and  purpose  of  an 
implementation  plan. 

In  Chapter  YL,  the  research  is  summarized,  and  conclusions  are  drawn.  Finally, 
recommendations  are  made  for  the  initial  deployment  and  for  further  research. 

Appendix  A  is  an  implementation  plan  for  the  initial  deployment  of  MAES.  Appendix 
B  is  a  basic  training  plan  and  includes  a  training  evaluation  form  to  be  used  by  the  deployment 
team.  Appendix  C  is  an  updated  version  of  the  MAES  user’s  manual  and  Appendix  D  is  a 
MAES  User  Survey  Form. 
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II.  IMPLEMENTATION  ISSUES  RESEARCH  FOR  TRADITIONAL 
AND  EXPERT  SYSTEMS  SOFTWARE 

This  chapter  discusses  the  potential  implementation  issues  involved  in  fielding 
software  applications  and  addresses  the  primary  research  question,  “What  are  the  potential 
implementation  issues  for  the  initial  fielding  of  MAES?”  Section  A  reviews  research  on 
implementation  issues.  Section  B  explains  the  foundations  of  a  successful  implementation 
using  the  Prerau  (1990)  implementation  model.  Section  C  examines  implementation  from  a 
user  and  management  perspective  following  the  Lucas-Ginzberg  (1990)  model.  Sections  D 
and  E  discuss  how  expert  systems  differ  from  traditional  information  systems  and  how  expert 
systems  can  be  categorized.  Sections  F  and  G  examine  variables  that  need  to  be  considered 
when  implementing  an  expert  system  and  the  relative  importance  of  implementation  factors 
depending  on  the  category  of  expert  ^stem.  Section  H  provides  a  summary  of  the  chapter. 
A.  SUMMARY  OF  IMPLEMENTATION  ISSUES  RESEARCH 

A  great  deal  of  research  has  been  done  on  the  different  factors  that  can  contribute 
towards  a  successful  implementation  of  an  information  system.  This  section  summarizes  the 
research  on  implementation  issues  that  was  most  applicable  to  this  thesis.  Multinovich  and 
Vlahovich  (1984)  identified  thirteen  steps  which  are  necessary  in  a  successfiil  implementation 
strategy.  They  determined  that  the  steps  are  interrelated  and  that  some  may  be  done 
concurrently.  Leonard-Barton  and  Deschamps  (1987)  concentrated  on  assessing  the  effect 
that  upper  management  can  have  in  the  implementation  of  new  technology.  Lucas  and 
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Ginzberg  (1991)  devoted  a  whole  book  to  the  development  and  testing  of  a  structural  model 
for  information  systems  implementation. 

Research  has  also  been  conducted  in  the  area  of  implementing  expert  systems.  Prerau 

authored  a  book  that  provides  a  step-by-step  approach  to  developing  and  managing  expert 

systems  (Prerau,  1991).  Bradley  and  Hauser  (1995)  provide  a  framework  for  ejq)ert  system 

implementation  based  on  categorizing  the  type  of  expert  system  that  is  being  implemented. 

B.  ,  PRERAU’S  METHODOLOGY  FOR  IMPLEMENTING  EXPERT  SYSTEM 
SOFTWARE  APPLICATIONS 

While  the  development  of  any  application  has  unique  aspects,  Prerau  maintains  that 
there  are  three  basic  phases  in  the  evolution  of  a  system  that  can  be  considered  generic: 
(Prerau,  1990) 

•  The  Initial  Phase  composed  of  project  start-up,  selection  of  the  domain  and 
selection  of  the  development  environment. 

•  The  Core  Development  Phase  consisting  of  the  development  of  a  feasibility 
prototype  system  and  foil  prototype  system. 

•  The  Final  Development  and  Deployment  Phase  composed  of  the  development  of 
a  production  system,  system  deployment,  and  system  operation  and  maintenance. 

According  to  Prerau,  ensuring  that  an  application  is  developed  using  the  above  basic  phases 

as  a  foundation  will  better  ensure  the  successful  development  and  deployment  of  a  new 

system.  The  reader  may  wish  to  review  Prerau’ s  discussion  of  the  Initial  and  Core 

Development  phases.  This  section  will  focus  on  the  third  phase  -  the  final  development  and 

deployment  phase  of  a  project. 


6 


In  the  final  phases  of  software  development,  Prerau  states  that  the  final  production 
system  is  built  and  the  system  is  deployed.  The  program  then  transitions  to  a  maintenance 
phase ,  with  program  and  knowledge  bugs  fixed  and  possibly  new  information  and  features 
added. 


1.  Development  of  a  Production  System 

According  to  Prerau,  the  decision  to  develop  a  production-quality  version  of  an 
application  fi-om  the  full  prototype  system  is  based  on  several  factors:  the  results  of  the  field 
tests  of  the  prototype  system;  feedback  fi'om  outside  experts,  potential  users,  and 
management;  and  estimates  of  the  cost  versus  benefits  of  the  final  system.  D'the  dedsion  is 
made  to  proceed,  then  the  development  of  the  production  system  is  initiated.  In  this  phase, 
the  fiill  prototype  is  made  into  a  program  that  is  robust,  reliable,  and  can  be  fielded.  Then  the 
system  is  deployed  and  put  into  long-term  operation.  To  produce  the  deployment  system, 
Prerau  suggests  several  issues  be  explored:  (Prerau,  1990) 

•  Possible  representation  and  re-implementation:  There  may  be  benefits  in  doing  a 
redesign  and  re-implementation  when  beginning  work  on  the  deployment  system. 
For  example,  when  it  is  decided  that  the  final  production  system  should  be 
rewritten  to  allow  it  to  be  run  on  new  hardware  or  with  a  new  software  tool,  the 
additional  cost  of  redesigning  the  representation  and  implementation  might  be 
lower. 

•  Deployment  hardware:  The  deployment  hardware  does  not  necessarily  have  to  be 
the  same  as  the  development  hardware,  although  sometimes  the  hardware  for 
development  was  selected  with  the  deployment  in  mind.  Some  of  the  fectors 
involved  in  hardware  selection  include:  reducing  hardware  cost,  making  program 
operation  and  maintenance  easier,  using  a  hardware  vendor  that  provides  better 
support,  and  putting  the  application  on  hardware  that  can  be  used  for  other 
purposes. 
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•  Deployment  software:  The  deployment  software  tool  does  not  have  to  be  the  same 
as  Ae  development  software  tool.  If  the  deployment  software  is  different,  the  cost 
of  converting  firom  the  development  software  will  have  to  be  considered. 

•  Input/Output  formats  and  mechanisms:  The  formats  and  mechanisms  for  user 
inputs  and  to  deliver  output  to  users  may  have  to  be  reconsidered.  Comments 
fi-om  the  users  involved  in  the  field  tests  and  from  domain  experts  may  aid  in  the 
design  of  good  input/output  formats. 

•  Testing:  The  production  system  should  be  tested  and  evaluated.  Validation  tests 
should  determine  that  the  system  effectively  solves  the  problem  for  which  it  was 
built,  at  an  acceptable  level  of  ejq)ertise.  Verification  tests  should  ensure  that  the 
expert  knowledge  acquired  has  been  correctly  implemented,  that  the  system  is 
operating  accurately  to  the  extent  required  and  is  as  free  as  possible  of 
programming  errors. 

•  Documentation:  The  documentation  that  will  accompany  the  system  must  be 
designed  and  written.  It  might  consist  of  printed  manuals,  on-line  help,  or  both. 
There  may  be  levels  of  documentation  for  system  maintainers,  system  operators, 
and  system  users. 

2.  System  Deployment 

When  the  production  system  is  completed  and  fully  tested,  the  system  can  be  deployed 
(Prerau,  1990).  There  are  several  factors  that  Prerau  (1990)  recommends  analyzing  before 
deployment: 

•  Mode  of  deployment:  There  are  several  options.  The  final  system  could  be 
delivered  to  users  as  a  stand-alone  system;  it  could  be  operated  as  a  separate  entity 
but  integrated  into  the  user’s  environment;  or  it  could  be  embedded  into  another 
system.  The  users  of  the  system  may  be  responsible  for  operating  it,  they  may 
have  responsibility  for  both  operating  and  maintaining  it,  or  they  may  utilize  the 
system  as  a  service. 

•  System  introduction:  If  the  users  have  been  clearly  identified  and  involved 
throughout  the  development,  introducing  the  system  into  the  working  environment 
may  not  pose  many  problems.  Users  to  whom  the  system  is  new  may  have  to  be 
convinced  to  change  their  present  mode  of  operation.  Introduction  of  a  new 
system  will  change  the  status  quo  and  may  arouse  the  fears  associated  with 
automation.  Beyond  these  standard  problems,  expert  systems  have  an  additional 
aspect  that  may  cause  problems  and  that  is  inherent  to  the  technology:  expert 
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systems  sometimes  make  mistakes.  Often  users  can  accept  incorrect  results  more 
ea^y  from  a  human  ejq)ert  than  from  a  computer  program.  When  a  system  makes 
mistakes  there  may  be  problems  getting  users  to  trust  the  system. 

•  User  training;  The  methods  to  be  used  for  training  system  operators  and  users 
must  be  decided.  Training  may  include  formal  courses,  training  manuals,  and/or 
on-line  tutorials. 

•  Documentation:  Any  documentation  for  system  operators,  maintainers,  or  users 
not  already  written  should  be  developed. 

Although  these  areas  are  directly  related  to  deployment,  many  of  the  issues  can  be  examined 
and  choices  made  vdiile  the  production  system  is  under  development  and  testing.  Many  issues 
may  be  resolved  earlier  in  the  project.  A  project  should  have  a  deployment  plan  specifying 
the  selected  deployment  mode,  hardware,  software,  pricing,  and  so  forth.  It  should  specify 
the  sequence  and  timing  of  events  that  will  be  followed  to  bring  about  the  initial  system 
deployment.  (Prerau,  1990) 

3.  System  Operation  and  Maintenance 

According  to  Prerau,  a  long  term  maintenance  group  should  be  formed  and  trained. 
Maintenance  not  only  includes  fixing  problems  found  during  system  operation,  but  also 
revising  internal  data  and  knovdedge  that  changes  over  time.  Since  maintenance  will  require 
not  just  changes  to  the  implementation  of  knowledge  but  also  changes  to  the  knowledge  itself, 
it  is  mandatory  that  a  domain  expert  be  involved  in  the  maintenance  process,  even  on  a 
consulting  basis  only.  A  decision  must  be  made  whether  to  have  centralized  maintenance, 
where  program  patches  and  revisions  come  from  a  single  source,  or  to  have  a  more 
distributed  maintenance  plan.  The  latter  approach  will  result  in  non-standard  versions  of  the 
system  and  may  not  be  desirable.  However,  it  does  allow  the  customization  of  the  software 
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to  fit  the  circumstances  of  each  site.  It  may  become  necessary,  during  the  lifetime  of  an 
expert  system,  to  make  major  changes  to  the  software  to  expand  its  scope  or  add  new 
features  and  capabilities.  System  revision  can  be  done  by  the  system  maintenance  group,  but 
if  the  expansion  is  large  enough,  it  may  be  necessaiy  to  use  an  independent  development 
group.  (Prerau,  1990) 

C.  IMPLEMENTING  A  SOFTWARE  APPLICATION 

Deployment  of  a  new  software  application  entails  bringing  such  a  ^stem  into 
operational  use  by  turning  it  over  to  the  end  user  (Multinovich  and  Vlahovich,  1984). 
Ultimately,  it  is  end  users  who  determine  the  success  of  a  system  through  their  use  of  the 
system.  The  deployment  process  can  be  considered  a  critical  success  factor  in  whether  a 
sof^are  application  is  accepted  by  the  end  users.  Lucas  and  Ginzberg  (1990)  developed  an 
implementation  model  based  on  previous  research  on  implementation  issues.  They  conclude 
that  the  most  consistent  relationships  with  software  success  or  Mure  have  been  demonstrated 
to  be  three  major  issues:  management  support,  user  involvement,  and  conduct  of  the 
implementation  process  itself 

1.  The  Manager  Model  for  Implementation 

The  Lucas  and  Ginzberg  model  consists  of  two  separable  submodels,  the  user  model 
and  the  manager  model.  The  manager  model  in  Figure  2-1  consists  of  the  following  variables: 
(Lucas  and  Ginzberg,  1990) 

•  Manager  acceptance:  This  variable  is  a  measure  of  the  extent  to  which  a  manager 
wants  a  particular  system  to  be  implemented. 
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Figure  2-1.  The  Lucas-Ginzbei^  manager  sub-model. 
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•  Manager  knowledge  of  system:  A  measure  of  how  well  a  manager  understands  the 
system  to  be  implemented.  Better  understanding  of  a  system’s  design  and 
capabilities  leads  directly  to  increased  acceptance. 

•  Manager  assessment  of  system  and  support:  Measures  the  manager’s  evaluation 
of  the  quality  of  the  system  and  its  supporting  mechanism.  Favorable  evaluations 
should  result  in  increased  acceptance. 

•  Manager  belief  in  system  concept:  The  extent  to  which  a  manager  believes  in  the 
underlying  concept  or  approach  behind  a  system  and  the  ability  of  that  approach 
to  solve  the  organization’s  decision  problems.  Stronger  belief  in  the  ^stem 
concept  will  result  in  greater  incentive  for  the  manager  to  become  involved  in 
system  development  and  leam  about  the  system. 

•  Top  management  support:  This  variable  measures  the  level  of  support  exhibited 
by  top  management  in  an  organization  for  the  use  of  a  particular  system.  Greater 
top  management  support  should  result  in  managers  being  more  willing  to  become 
involved  in  system  development  and  having  greater  belief  in  the  system  concept. 

According  to  Lucas  and  Ginzberg,  an  implementation  process  begins  with  management 

initiation  and  acceptance,  so  the  management  issues  are  usually  addressed  first.  A  ^stem  that 

is  accepted  by  management  still  stands  a  chance  of  failure  if  the  users  do  not  accept  it.  But 

ensuring  that  the  above  variables  are  addressed  and  optimized  increases  the  probability  of  a 

successful  deployment.  (Lucas  and  Ginzberg,  1990) 

2.  The  User  Model  for  Implementation 

One  important  objective  of  a  system  is  to  improve  the  user’s  job  performance. 
Therefore  the  Lucas  and  Ginzberg  user  model  begins  with  the  user’s  personal  stake  in  the 
system  and  ends  with  satisfaction  and  performance.  The  user  model  in  Figure  2-2  consists 
of  the  following  variables:  (Lucas  and  Ginzberg,  1990) 

•  User  Acceptance:  This  variable  measures  the  potential  user’s  predisposition  to 
personally  use  a  specific  system.  It  is  a  measure  of  behavioral  intention  that  will 
be  reflected  in  actual  use. 
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Manager  Model 


Figure  2-2.  The  Lucas-Ginzberg  user  sub-model. 
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Characteristics  Characteristics  Demographics 


•  User  knovdedge  of  system:  This  variable  measures  how  much  the  user  understands 
the  functioning  of  a  system.  Better  knowledge  of  a  system’s  design  and 
capabilities  leads  to  increased  acceptance. 

•  User  assessment  of  system  and  support:  Measures  the  user’s  evaluation  of  the 
system  and  its  supporting  mechanisms. 

•  System  characteristics:  This  variable  represents  the  features  abd  capabilities  of  the 
system.  Examples  of  these  characteristics  may  be  ease  of  use  or  the  fit  between 
system  capabilities  and  the  demands  of  the  user’s  job.  Easy  to  use  systems  which 
meet  the  user’s  needs  are  more  likely  to  be  accepted. 

•  User-researcher  involvement:  Indicates  the  degree  of  interaction  between  a  user 
and  the  system  deagner.  Greater  involvement  leads  to  greater  user  knowledge  of 
the  system  capabilities. 

•  User  perception  of  management  support:  One  of  the  key  determinants  of  the  user’s 
acceptance  is  the  manager’s  acceptance  or  support  of  the  system.  It  is  the  user’s 
perception  of  management  support,  not  an  actual  measurement. 

•  User  knowledge  of  system  purpose  and  use:  Without  knowledge  of  system 
purpose  or  use,  the  user  will  be  unable  to  assess  the  importance  of  the  system. 

•  Problan  urgency:  The  greater  the  user’s  perception  of  problem  urgency,  the  more 
important  a  system  addressing  that  problem  and  the  greater  the  user’s  stake  in  that 
problem. 

•  Organizational  support:  The  degree  to  which  the  organization  provides  the 
environment  and  facilities  to  make  access  to  and  use  of  the  system  easy. 

A  ^stem  deployment  can  be  viewed  as  an  ongoing  process.  It  continues  as  long  as 
new  users  are  bang  introduced  to  the  system.  The  concept  of  introducing  a  system  into  the 
existing  user  operation  without  major  upheavals  will  require  following  both  Prerau’s  (1990) 
implementation  methodology,  while  at  the  same  time  paying  close  attention  to  the  controllable 
variables  in  the  Lucas-Ginzberg  implementation  model  of  introducing  a  system  to  the  user. 
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Prerau  states  that  for  a  successful  deployment  of  an  expert  system,  there  must  be 
clearly  defined  goals  for  the  fielding  (Prerau,  1990).  From  the  discussion  of  the  previous 
sections,  these  goals  can  be  summarized  as  follows: 

•  Obtaining  management  support.  (Prerau,  1990  and  Lucas-Ginzberg,  1990) 

•  Involving  the  users  in  the  development.  (Prerau,  1990  and  Lucas-Ginzberg,  1990) 

•  Involwng  the  users  in  the  evaluation  of  the  prototype.  (Prerau,  1990) 

•  Proper  training  of  the  users.  (Prerau,  1990  and  Lucas-Ginzberg,  1990) 

•  Establishing  effective  lines  of  communication  between  the  users  and  the 
maintenance  team.  (Prerau,  1990) 

•  Conducting  a  post-deployment  evaluation.  (Multinovich  and  Vlahovich,  1984) 
The  latter  sections  in  this  chapter  examine  the  specifics  of  meeting  the  goals  for  successMy 
deploying  a  software  application.  The  following  section  discusses  the  differences  between 
an  expert  system  and  a  traditional  information  system. 

D.  DIFFERENCES  BETWEEN  AN  EXPERT  SYSTEM  AND  A  TRADITIONAL 

INFORMATION  SYSTEM 

Expert  systems  differ  fi-om  traditional  management  information  systems  in  a  number 
of  ways.  These  differences  include  the  type  of  problem-solving  approach,  the  source  and 
acquisition  of  system  specifications,  and  the  physical  design  of  the  system.  First,  expert 
systems  employ  a  heuristic  approach  rather  than  the  algorithmic  approach  used  by  traditional 
information  systems.  Second,  expert  systems  are  designed  to  provide  a  solution  to  a  problem 
based  solely  on  the  knowledge  of  one  or  more  experts.  This  can  limit  the  degree  of  user 
involvement  in  the  design  process  because  the  source  of  the  system’s  knowledge  comes 
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from  an  expert.  The  reliance  on  the  talent  and  skill  of  a  knowledge  engineer  is  also  a  major 
difference  from  traditional  system  requirements.  (Bradley  and  Hauser,  1995) 

Another  difference  in  the  development  process  lies  in  who  physically  controls  the 
development.  Traditional  information  system  development  has  been  handled  by  the  MIS 
department.  Expert  system  development,  however,  is  different  enough  that  management  is 
often  uncertain  as  to  the  level  of  involvement  of  the  MIS  department  in  the  development 
process.  Expert  systems  are  often  developed  to  diagnose  mechanical  problems  in  complex 
machinery  where  technicians,  such  as  industrial  engineers,  are  used  in  the  development 
process  since  they  understand  the  problems  and  are  comfortable  with  computers  as  tools. 
This  approach  can  result  in  a  bypass  of  traditional  system  development  life  cycle  requirements. 
Surveys  of  expert  systems  found  that  more  than  50%  were  developed  outside  the  MIS 
department.  (Bradley  and  Hauser,  1995). 

Expert  systems  possess  unique  characteristics  that  distinguish  them  from  traditional 
management  information  systems.  Expert  systems  also  vary  from  one  another.  The  following 
subsections  discuss  the  differaices  between  expert  systems  and  how  the  previously  discussed 
variables  from  the  Lucas-Ginzberg  model  and  the  Prerau  development  process  affect  expert 
system  development. 

E.  CATEGORIES  OF  EXPERT  SYSTEMS 

Expert  systems  possess  distinct  attributes  that  differentiate  them  from  one  another. 
If  such  differences  exist  between  expert  systems,  then  it  is  possible  that  a  single 
implementation  plan  would  not  be  appropriate  for  all  expert  systems.  Therefore,  the  first  step 
in  expert  system  implementation  planning  should  include  categorizing  the  system  according 
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to  its  attributes  and  developing  an  implementation  plan  based  on  those  attributes.  Expert 
systems  can  be  broken  down  into  four  categories  based  on  the  complexity  of  the  knowledge 
they  contain  and  the  level  of  computing  technology  used  to  run  them  (Meyer  and  Curley, 
1991).  Meyer  and  Curley  developed  a  classification  scheme  (Figure  2-3)  that  views  expert 
systems  along  these  two  attributes. 
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Within  the  expert  system  classification  scheme,  a  system  can  be  classified  into  one  of  four 
gropps:  (Bradley  and  Hauser,  1995) 

•  Category  I:  Low  to  mid  knowledge  complexity;  Low  to  mid  technical  complexity. 
These  ^sterns  are  developed  in  a  problem  domain  that  contains  highly  structured 
knowledge  that  can  be  ejq)ressed  in  IF-THEN  rules.  They  enhance  the 
productivity  of  a  very  small  group  of  users  who  focus  on  a  narrow  problem,  or  a 
large  group  of  users  who  make  simple,  repetitive  decisions.  These  systems  are 
often  developed  as  a  PC  based  applications  using  a  relatively  inexpensive  expert 
system  shell. 

•  Category  11:  Mid  to  high  knowledge  complexity;  Low  to  mid  technological 
complexity.  This  type  of  expert  system  is  also  typically  implemented  using  an 
expert  system  shell.  These  systems  address  problems  in  more  complex  problem 
domains  and  require  elaborate  reasoning  in  specialized  fields.  The  knowledge 
required  for  these  systems  is  much  more  complex  and  usually  involves  true 
expertise  instead  of  commonly  used  procedures. 

•  Category  DI;  Low  to  mid  knowledge  complexity;  Md  to  high  technical 
complexity.  These  ejq)ert  systems  address  problems  in  highly  structured,  simple 
domains.  The  technology  required  to  implement  the  system  involves  special  user 
interfeces,  complex  database  interactions,  and  usually  a  large  programming  effort. 
These  expert  systems  require  the  technical  knowledge  and  skill  of  a  computer 
scientist  for  development. 

•  Category  IV:  Md  to  high  knowledge  complexity;  Md  to  high  technical 
complexity.  This  type  of  expert  system  is  a  combination  of  types  n  and  HI.  The 
combination  of  complex  knowledge  and  advanced  technologies  requires  a 
development  team  of  trained  knowledge  engineers  and  computer  scientists.  The 
development  effort  for  these  ^sterns  usually  includes  complex  programming,  data 
base  management,  and  systems  integration  as  well  as  an  extensive  knowledge 
acquisition  effort. 

Knowing  that  expert  systems  differ  not  only  fi-om  traditional  management  information  systems 
but  also  from  each  other  can  prove  valuable  in  planning  an  implementation. 

F.  VARIABLES  THAT  AFFECT  EXPERT  SYSTEM  IMPLEMENTATION 

In  addition  to  the  implementation  factors  previously  addressed  in  this  chapter,  the 
unique  nature  of  expert  systems  cany  with  them  additional  considerations  for  implementation. 
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Ejq)ert  systems  often  require  anployees  to  cease  their  current  pattern  of  behavior,  learn  a  new 
pattern  of  behavior  that  incorporates  the  innovation,  and  then  persist  in  the  new  behavior. 
Implementation  of  an  expert  system  involves  modifying  an  individual’s  behavior  to  include 
use  of  the  erqrat  system.  Bradley  and  Hauser  (1995)  identified  a  number  of  factors  that  can 
influence  user  acceptance  of  an  expert  system.  The  factors  are  organized  into  three  mmn 
groups:  iimovation  characteristics,  organizational  influences,  and  individual  differences. 

1.  Innovation  Characteristics 

In  expert  system  implementation,  user  attitudes  may  be  influenced  by  attitudes 
towards  computers  in  general.  Prior  expectations  about  the  expert  system  have  also  been 
shown  to  be  an  important  factor  in  successful  implementation.  Ginzberg  (1981),  found 
evidence  that  when  users  hold  unrealistic  prior  expectations  about  a  system,  they  are  more 
resistant  to  change  and  the  implementation  is  more  Ukely  to  fail. 

2.  Organizational  Influences 

Leonard-Barton  found  that  organizational  influences  impact  on  user  acceptance 
(Leonard-Burton,  1987).  Later  sections  of  this  chapter  discusses  a  number  of  studies  that 
have  concentrated  on  the  effects  of  organizational  influence  on  implementation  success. 
Organizational  influence  factors  include,  but  are  not  limited  to,  user  participation  in 
development,  rewards  for  using  the  system,  user  training,  and  formal  and  informal 
management  support.  (Bradley  and  Hauser,  1995) 

3.  Individual  Differences 

Individual  differences,  such  as  user  education  and  experience,  have  been  shown  to 
have  a  significant  effect  on  implementation.  A  key  factor  in  the  personal  characteristics  of  the 
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user  is  their  attitude  towards  change.  The  extent  to  which  a  user  perceives  a  change  to  then- 
job  oTvironment  will  often  effect  the  amount  of  resistance  they  exhibit.  The  effect  of  personal 
characteristics  on  implementation  is  dependent  upon  the  perceived  change  caused  by  the 
expert  system.  (Bradley  and  Hauser,  1995) 

G.  RELATIVE  IMPORTANCE  OF  EXPERT  SYSTEM  IMPLEMENTATION 

FACTORS 

Bradley  and  Hauser  (1995)  suggest  that  any  ejqpert  system  implementation  plan  should 
include  a  careful  consideration  of  fectors  in  the  three  categories:  innovation  characteristics, 
organnational  differaices,  and  individual  differences.  The  relative  significance  of  each  factor, 
however,  will  vary  as  a  result  of  the  type  of  expert  system  being  implemented.  Table  2-1 
represents  each  factor  with  a  High,  Medium,  or  Low  value  to  indicate  the  degree  of  emphasis 
the  fector  should  have  in  an  implementation  plan  based  on  the  classification  of  expert  system. 
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Factor 

Expert  System  Classification 

T  n  m  TV 

Innovation  Characteristics 

User  perception  of  computing  technology 

H 

H 

H 

H 

User  perception  of  expert  systems 

L 

M 

M 

H 

Organizational  influences 

User  involvement  in  expert  system  design 

H 

H 

H 

H 

User  involvement  in  expert  system  implementation 

H 

H 

H 

H 

Reward  structure 

L 

M 

M 

H 

Training 

L 

M 

M 

H 

Top  management  support 

L 

M 

M 

H 

Supervisor  support 

H 

H 

H 

H 

Advocates 

L 

M 

M 

H 

Consulting  aids 

L 

M 

M 

M 

Individual  Differences 

User  perception  of  change 

H 

H 

H 

H 

L  =  low  emphasis  in  the  implementation  plan 
M=  medium  emphasis  in  the  implementation  plan 
H  =  high  emphasis  in  the  implementation  plan 


Table  2-1.  Relative  Importance  of  Expert  System  Implementation  Factors. 
(Bradley  and  Hauser,  1995) 


The  above  framework  suggests  that  a  single  implementation  methodology  may  not  be 
appropriate  for  all  expert  systems.  Expert  system  developers  should  categorize  the  expert 
system  early  in  the  analysis  of  the  system  and  develop  an  implementation  plan  that  emphasizes 
what  is  most  ^propriate  for  that  system.  In  a  resource  constrained  project,  this  method  can 
also  be  used  to  determine  where  resources  should  optimally  be  concentrated  to  better  ensure 
successful  system  implementation.  (Bradley  and  Hauser,  1995) 
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H.  SUMMARY 


In  this  chapter  the  Prerau  (1990)  methodology  for  implementing  software  applications 
was  examined  as  a  proven,  practical  approach  for  developing  and  managing  the 
implementation  of  expert  systems.  Additionally,  the  Lucas-Ginzberg  (1990)  model  for 
implementation  was  examined  as  it  provides  both  the  user’s  and  management’s  perspective 
towards  implementation.  Finally,  implementation  issues  that  are  specific  to  expert  systems 
were  addressed  and  found  to  be  dependent  on  the  type  of  expert  system  being  implemented. 
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III.  IMPORTANT  IMPLEMENTATION  ISSUES 

This  chapter  provides  the  background  information  needed  to  address  the  primary  and 
subsidiary  research  questions  listed  in  Chapter  I.  Sections  A,  B,  and  C  examine  methods  of 
obtaining  management  support,  involving  the  user  in  the  development,  and  training  the  users, 
respectively.  Sections  D  and  E  discuss  establishing  effective  communications  and  conducting 
a  post-implementation  evaluation.  Section  F  analyzes  hardware  concerns  and  Section  G 
provides  a  summary  of  the  chapter. 

A.  MANAGEMENT  SUPPORT 

This  section  examines  the  research  conducted  on  the  importance  and  methods  of 
obtaining  management  support  and  also  methods  of  overcoming  the  resistance  of  management 
to  adopt  a  new  system. 

1.  The  Importance  of  Obtaining  Management  Support 

Since  managers  and  users  interact  on  a  regular  basis,  the  manager’s  attitude  toward 
a  system  is  an  important  fector  for  deployment  success.  Management  support  has  been 
identified  by  a  number  of  researchers  as  a  key  factor  of  deployment  success  (e.g.  Ginzberg 
(1981),  Multinovich  and  Vlahovich  (1984),Leonard-Barton  and  Deschamps  (1988),  Bouldin, 
(1989),  Prerau,  (1990),  Lucas-Ginzberg  (1990)).  Their  major  findings  were: 

•  Management  commitment  and  support  are  critical  and  essential  for  successfiil 
implementation  of  MIS/DSS.  (Ginzberg,  1981) 

•  Capital  resources  required  for  system  implementation  increase  significantly,  and 
managptmfint  must  approve  these  expenditures.  (Multinovich  and  Vlahovich,  1984) 
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•  Subordinates  are  much  more  prone  to  act  when  their  superiors  are  interested  in 
the  outcome.  (Multinovich  and  Vlahovich,  1984) 

•  Perceptions  of  managerial  behavior  supporting  or  directly  urging  use  of  an 
innovation  are  positively  related  to  use.  (Leonard-Barton  and  Deschamps,  1988) 

•  Obtaining  upper  management  approval  is  the  point  in  the  implementation  beyond 
which  you  cannot  proceed  without  their  consent.  (Bouldin,  1989) 

•  Obtaining  management  approval  is  one  of  the  key  steps  in  the  initial  phases  of  an 
implementation  plan.  (Prerau,  1990) 

•  One  of  the  key  determinants  of  the  user’s  personal  stake  is  his  or  her  manager’s 
acceptance  or  support  of  the  system.  (Lucas-Ginzberg,  1990) 

For  most  software  implementations,  this  will  be  a  critically  important  step  of  the  deployment 

process.  Management  support  is  important  because  it:  (Multinovich  and  Vlahovich,  1984) 

•  Provides  adequate  resources. 

•  Provides  psychological  support. 

•  Creates  a  climate  that  acknowledges  a  problem  exists. 

In  the  Navy,  a  fector  which  is  very  applicable  to  the  deployment  process  is  that  subordinates 
are  much  more  prone  to  act  when  their  superiors  are  interested  in  the  outcome  ^Multinovich 
and  Vlahovich,  1984). 

2.  Methods  of  Gaming  Support  and  Overcoming  Resistance 

Multinovich  and  Phatak  (1981)  conducted  research  to  discover  methods  of  obtaining 
management  support.  They  propose  several  methods  to  gain  support  fi-om  upper 
management: 

•  “Sell”  the  system  to  management.  Management  must  be  apprized  of  the  benefits 
of  the  ^stem  and  convinced  that  it  would  be  to  their  advantage  to  support  and 
implement  the  system.  Implementation  of  any  new  technology  is  in  essence  a 
marketing  task  (Leonard-Barton,  1987). 
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•  Create  the  illusion  of  support  or  generate  support  by  sending  copies  of  reviews  to 
top  management,  particularly  action  memos.  Others  not  aware  of  middle 
managemait’s  support  may  be  led  to  believe  top  management  supports  the  system 
and  will  in  turn  support  it. 

•  Ensure  that  top  management  reads  the  memos.  By  reading  the  memos,  they  may 
change  their  opinion  and  support  the  system. 

Bouldin  also  developed  some  methods  for  overcoming  management  resistance:  (Bouldin, 
1989) 


•  Conduct  a  marketing  blitz  to  ensure  that  management  has  some  basic  level  of 
understanding  about  the  software  and  its  usefulness.  Some  methods  to  achieve  this 
heightened  awareness  are:  Utilize  email,  show  videotapes  of  the  software  in  action, 
give  formal  and  ad  hoc  demonstrations  and  presentations,  and  utilize  your  informal 
network. 

•  Conduct  demonstrations  to  management  to  show  off  the  capabilities  and 
advantages  of  the  new  software. 

•  Gain  the  support  of  technical  experts  whose  opinion  is  trusted  by  management  and 
have  them  available  for  testimony  during  presentations. 

•  Publicize  the  results  of  a  fevorable  cost-benefit  analysis. 

Obtaining  the  tadt  support  of  management  is  not  enough.  Management  must  indicate 
through  their  words  and  actions  that  they  support  the  system  (Lucas  and  Ginzberg,  1990). 
Actions  that  show  support  include:  (Lucas  and  Ginzberg,  1990) 

•  Attending  meetings  with  the  users  about  the  system. 

•  Providing  time  for  the  users  to  train  on  the  system. 

•  Becoming  knowledgeable  about  the  basics  of  the  system. 

•  Personally  using  the  system. 
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In  order  for  management  to  actually  participate  in  the  above  actions,  they  must  first  be 
convinced  of  the  benefits  of  the  system.  Benefits  that  will  likely  appeal  to  management 
include:  (Multinovich  and  Phatak,  1981) 

•  Saving  money:  May  require  a  cost-benefit  analysis. 

•  Improving  Quality  of  Life:  An  increase  in  fi'ee  time  may  result  because  of  a 
possible  decrease  in  the  workload. 

•  Using  the  system  as  a  training  tool:  An  intangible  benefit  that  may  be  hard  to 
quantify. 

Benefits  such  as  those  listed  above  would  go  a  long  way  in  convincing  management  that  it 
would  be  to  their  advantage  to  support  and  implement  the  system.  Management  support  is 
so  necessary  that  Multinovich  and  Vlahovich  (1984)  recommend  aborting  any  fiuther 
development  of  a  system  if  management  support  is  lacking.  Continuing  the  development  in 
such  cases  would  be  a  waste  of  time  and  money. 

3.  Cost-Benefit  Analysis  as  a  Method  of  Obtaining  Management  Support 

The  purpose  of  a  cost-benefit  analyas  is  to  assess  as  accurately  as  possible  the  benefits 
and  costs  associated  with  implementing  a  software  application.  The  analysis  of  benefits  must 
include  determining  both  tangible  and  intangible  benefits.  Tangible  benefits  include  things  that 
can  be  measured  such  as  reduction  in  staff  size,  reduced  maintenance  costs,  and  increased 
handling  capacity.  Intan^ble  benefits  include  things  that  are  difficult  to  measure  such  as 
improving  employee  morale  and  increased  training  opportunities.  (Powell,  1993) 

After  the  analysis  has  been  completed,  the  results  must  be  analyzed  carefully. 
Although  management  may  not  ever  see  the  exact  figures,  it  is  important  to  be  conservative. 
The  main  reason  that  an  analysis  is  done  is  to  enable  a  person  to  address  whether  or  not  a  new 
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system  will  be  a  substantial  improvement.  The  quantification  will  bring  some  objectivity  to 
an  otherwise  subjective  issue.  Management  will  feel  more  comfortable  if  the  advocate  feels 
confident  in  promoting  a  new  system.  If  the  estimates  indicate  a  promising  area  for 
productivity  improvement,  the  figures  can  be  used  as  a  basis  for  obtaining  the  support  of 
management.  (Bouldin,  1989) 

B.  USER  INVOLVEMENT  IN  THE  DEVELOPMENT 

Research  has  shown  that  users  should  actively  be  involved  in  the  development  of  a 
system  through  its  life  cycle  (Ginzberg,  (1978),  King  (1979),  Madni  (1988),  and  Lucas- 
Ginzberg  (1990)).  The  key  findings  fi’om  their  research  include: 

•  A  new  system  should  satisfy  a  particular  user’s  requirements  without  adversely 
affecting  the  information  requirements  of  other  users  in  the  system.  (King,  1979) 

•  Greater  involvement  of  the  user  in  the  development  should  lead  to  greater  user 
knowledge  of  the  system’s  capabilities.  (Lucas-Ginzberg,  1990) 

•  Rq)orts  should  be  made  available  to  users  if  pertinent.  This  helps  them  to  feel  they 
are  a  part  of  the  implementation  and  encourages  involvement.  (Ginzberg,  1978) 

•  The  single  most  important  human  factors  issue  is  understanding  the  requirements 
of  the  intended  user.  (Madni,  1988) 

User  participation  must  not  be  token  nor  given  lip  service.  It  must  be  meaningful  and 
genuine.  Enlisting  the  aid  of  the  users  in  the  development  of  software  is  prudent  since  the 
users  often  know  much  more  than  can  be  learned  through  interviews  or  surveys.  Users  have 
often  been  through  solutions  that  did  not  work  and  may  have  observed  solutions  that  could 
have  worked  if  the  users  had  had  some  type  of  input  to  the  design  (Multinovich  and 
Vlahovich,  1984). 
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The  knowledge  by  a  user  that  his  suggestion  was  implemented  in  the  design  of  an  MIS 
stimulates  his  enthusiasm  for  the  system.  Users  should  therefore  be  made  to  feel  that  they  are 
contributing  to  a  system,  no  matter  how  small  the  suggestion.  This  not  only  helps  the  user 
develop  realistic  ejq)ectations  about  the  system’s  capabilities,  but  it  can  also  result  in  positive 
feelings  of  self-worth  and  ownership  that  decrease  user  resistance  to  change  and  commit  the 
users  to  the  system.  (Bradley  and  Hauser,  1995) 

It  must  be  understood  that  user  participation  is  not  a  total  solution  but  only  one  part 
of  a  successful  implementation.  There  are  concerns  which  should  be  kept  in  mind.  Users  are 
not  professional  developers,  and,  without  professional  assistance,  they  tend  to  miss 
opportunities  for  improvement  and  the  applications  of  appropriate  new  information 
technologies.  Since  users  do  not  generally  think  in  terms  of  the  whole  process,  their  redesign 
ideas  may  be  limited  or  flawed.  Strong  proponents  for  users  participating  in  system  design 
increasingly  warn  that  it  can  lead  to  a  form  of  “incrementalism,”  rather  than  any  appreciable 
performance  improvements.  Even  when  users  have  good  ideas,  there  still  lies  the 
communication  barrier  that  can  exist  between  the  us«-s  and  the  developers.  Developers  often 
have  dfficulty  understanding  why  a  user  desires  a  certain  feature  especially  if  it  conflicts  with 
the  developer’s  concept  of  how  the  system  can  or  should  be  designed.  Greater  participation 
by  users  is  not  a  total  solution,  but  that  is  not  saying  that  their  participation  has  no  value. 
Participation  is  necessary,  but  not  sufficient  for  good  system  design  concepts.  Developers’ 
unaided  perceptions  about  users’  jobs  are  often  wrong.  In  conclusion,  users  and  developers 
must  design  a  system  together.  (Markus  and  Keil,  1994) 
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C.  USER  TRAINING 


This  section  discusses  the  importance  and  contributions  of  user  training  towards  a 
successful  implementation.  Determining  training  needs  and  examination  of  effective  training 
methods  will  be  addressed  as  factors  in  formulating  an  effective  training  plan.  Finally, 
concluaons  will  be  drawn  based  on  available  information  on  user  training  as  it  contributes  to 
the  success  of  software  implementation. 

1.  The  Importance  of  Training 

Although  user  interfeces  are  designed  to  be  easy  to  use  and  prompt  the  user  through 
system  functions,  training  is  still  required.  This  is  especially  so  if  the  user  has  minimal 
experience  using  computers  or  expert  systems.  A  successful  training  program  will  convince 
any  skeptics  that  the  system  is  both  crucial  and  necessary.  A  good  initial  impression  of  an 
application  can  be  established  by  means  of  effective  training.  Properly  trained  users  will  have 
a  superior  comprehenaon  and  awareness  of  the  project.  Without  proper  training,  the  success 
of  an  application  can  be  at  risk  because  training  ^ves  the  end-users  the  required  skills  and 
knowledge  to  reach  anticipated  levels  of  productivity.  Training  also  produces  greater 
confidence.  This  translates  into  the  end-user  referring  to  the  computer  as  “my  computed’ 
instead  of  “the  computer.”  As  a  result,  trained  users  will  use  an  application  more  efficiently 
and  effectively,  decreasing  their  dependence  on  support.  Computers  and  e?q)ert  systems  are 
worth  little  if  the  people  operating  them  can  not  use  them  properly.  Therefore,  user  training 
should  be  considered  a  critical  success  factor  in  the  successful  deployment  of  software. 
(Holek,  1994) 
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Expecting  users  to  be  computer  literate  is  not  an  assumption  that  can  always  be  made. 
Some  people  are  computer  illiterate,  suffer  from  computer  phobia,  or  are  flgain.«;t  changes  in 
the  workplace.  Also,  just  knowing  how  to  use  a  computer  does  not  guarantee  that  a  person 
can  operate  a  spreadsheet  package,  word  processor,  or  an  expert  system.  Users  must  be 
trained  to  use  the  systems  they  will  use  in  their  line  of  work.  Not  only  on  a  basis  of  what 
button  to  press,  but  on  a  higher  level,  so  that  users  can  do  some  troubleshooting  on  their  own, 
and  are  able  to  at  least  assess  what  the  problem  is  and  contact  the  right  person  for  help.  User 
training  will  take  time,  effort,  and  money.  (Nelson,  1995) 

The  questions  that  need  to  be  addressed  are: 

•  Who  should  be  trained? 

•  How  much  training  should  be  done? 

•  Which  methods  should  be  used  for  training  the  users? 

•  Who  should  do  the  training? 

Devising  the  training  plan  to  answer  the  above  questions  correctly  will  contribute  to  a 
successful  deployment. 

Lastly,  the  training  for  a  software  application  will  not  merely  mean  teaching  a  user 
how  to  input  data  or  how  to  get  answers  from  the  computer.  Training  also  means  educating 
the  user  in  the  overall  purpose  of  the  system:  what  it  is  supposed  to  do,  how  does  it  do  it, 
why  do  we  want  or  need  it  done,  and  who  must  do  it.  Effective  training  does  not  just  tell  the 
users  how  important  their  contributions  are,  it  lets  them  see  that  they  are  an  important, 
integral  part  in  the  entire  process  and  that  the  success  of  the  implementation  depends  on  how 
well  they  learn  what  is  being  taught.  (Multinovich  and  Vlahovich,  1984) 
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2.  Training  Plan  Formulation 

A  properly  devised  training  plan  will  ensure  a  successful  deployment.  The  software 
should  have  already  been  demonstrated  to  potential  users  via  presentations  and 
demonstrations  that  have  informed  them  about  its  capabilities  and  relevance  to  them.  The 
actual  trainii^  should  provide  sufficient,  detailed  information  and  hands  on  experience  to  use 
the  new  system  effectively  (Bouldin,  1989).  A  framework  for  building  an  effective  training 
plan  includes:  (Nelson,  1995) 

•  Assessment  of  training  needs 

•  Development  of  training  materials  and  methods 

•  Training  of  trainers 

•  Preliminary  evaluation  of  training  methods  and  materials 

•  Formal  training  and  learning 

•  Evaluation  of  training 

By  paying  proper  attention  to  filling  in  the  details  of  the  above  steps,  training  will  result  in 
users  who  are  able  to  operate  the  system  effectively  and  efficiently,  who  know  who  to  contact 
in  the  event  of  a  problem,  and  who  are  aware  of  the  capabilities  and  purpose  of  the  system. 
The  following  methodology  developed  by  Nelson  (1995)  was  used  to  develop  the  training 
plan  in  Appendix  B  of  this  thesis. 

a.  Assessment  of  Tridning  Needs 

In  assessing  training  needs,  it  must  be  determined  who  to  train,  what  kind  of 
skills  need  to  be  trained,  what  kind  of  methods  of  trmning  to  use,  and  who  is  to  carry  out  the 
training.  On  the  level  of  the  user,  the  deployment  team  must  decide  what  kinds  of  computer 
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skills  each  person  needs  to  be  able  to  perform  his  or  her  tasks  efficiently  and  productively 
(Prerau,  1990).  An  assessment  of  what  systems  or  pieces  of  software  each  individual  needs 
to  be  able  to  use  will  be  done.  A  dedsion  on  whether  the  user  needs  general  use-of-computer 
training  or  only  system  specific  instruction  has  to  be  made.  The  opinion  of  each  individual 
shoidd  be  heard  with  r^ards  to  interests,  actual  needs  and  preferences.  On  the  level  of  tasks 
to  be  performed  within  an  organization,  an  assessment  of  what  are  those  tasks  and  who 
should  be  trained  to  perform  those  tasks  needs  to  be  made.  A  good  practice  is  to  ensure  that 
all  relevant  personnel  become  qualified  to  use  the  new  software.  As  with  any  computer 
system,  the  organization  should  have  its  expert  users,  and  each  user  who  needs  to  use  the 
software  should  possess  adequate  skills.  (Nelson,  1995) 

Within  the  last  three  years,  personal  computers  have  become  increasingly 
available  to  business  and  government  organizations.  Even  with  the  widespread  use  of 
personal  computers,  training  must  address  the  user  who  may  have  no  to  little  experience  in 
using  personal  computers.  The  training  may  therefore  have  to  include  the  basics  of  installing 
the  software  and  accessing  the  program  from  the  operating  system.  (Leinonen,  1995) 

Training  needs  to  be  given  to  everyone  associated  with  the  system.  This 
requirement  includes  the  following  personnel:  (Holek,  1994) 

•  Upper  management;  Acquaint  upper  management  with  the  capabilities  and 
limitations  of  the  system. 

•  Line  management:  This  training  will  enable  line  management  to  understand  how 
the  users  input  data  to  the  system  and  how  they  use  the  outputs.  The  training  will 
allow  line  management  to  understand  what  the  software  does  and  how  it  does  it. 
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•  Users:  This  is  an  opportunity  to  stress  to  the  users  the  advantages  that  the  system 
has  to  offer  them.  The  goals  of  this  portion  of  training  is  to  provide  the  users  with 
the  necessary  skills  to: 

-  Input  data  into  the  system. 

-  Use  the  output  from  the  system. 

-  Utilizing  all  the  embedded  help  functions. 

-  Keep  the  system  performing  the  way  it  was  meant  to. 

It  should  be  stressed  that  training  should  not  only  teach  personnel  how  to  operate  the 
software  but  also  teach  persoimel  about  the  purposes  of  the  software. 

Based  on  the  different  audiences  that  the  deployment  team  will  be  training, 
different  methods  of  training  will  have  to  be  used.  Training  can  be  arranged  into  four 
different  categories:  (Cousins,  1994) 

•  Seminar  style  presentation:  Provide  trainees  with  a  basic  understanding  of  the 
operation  and  purposes  of  the  new  software. 

•  Classroom/Hands-on:  Provide  trainees  with  an  introduction  to  the  capabilities  and 
purpose  of  the  new  software.  In  addition,  users  will  receive  instruction  on  the 
advanced  operations  of  the  software,  e.g.,  data  input,  following  troubleshooting 
paths,  using  online  help,  etc. 

•  On-The-Job  training:  The  objective  of  this  type  of  training  is  to  have  intermediary 
trainers  provide  training  to  users  who  missed  the  initial  training  or  entered  their 
poation  after  the  software  was  introduced.  This  type  of  training  can  also  be  used 
in  the  event  of  software  upgrades  or  additions. 

•  Follow-up  Sessions:  The  objective  of  this  phase  of  the  training  will  be  to  monitor 
the  user’s  performance  with  the  new  software.  The  duration  should  be 
approximately  once  every  three  months  and  not  last  longer  than  one  hour. 

The  users  and  line  management  should  attend  classroom  presentations  to  teach  them  about 

the  benefits  and  purpose  of  a  software  application.  The  users  remain  at  the  training  site  to 

receive  the  hands-on  portion  of  the  training.  A  seminar  presentation  can  be  given  to  the  upper 

management  in  conjunction  with  a  follow-up  visit  to  each  department  in  the  organization. 
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These  four  types  of  training  will  ensure  that  everyone  involved  with  the  new  software  receives 
training  that  is  tailored  to  their  specific  needs  and  interests.  (Cousins,  1994) 

b.  Development  of  Training  Materials  and  Methods 

The  actual  training  may  consist  of  a  combination  of  the  four  different 
categories  of  training.  A  training  plan  should  contain  the  following  elements;  (Leinonen, 
1995) 

•  The  purpose  and  objectives  of  the  training 

•  The  required  training  materials 

•  A  timeline  of  training  events 

•  A  detailed  outline  of  what  will  be  covered 

•  An  evaluation  form  for  the  trainees  to  complete  on  the  effectiveness  of  the  training 
The  objective  of  the  finished  training  plan  is  to  ensure  that  the  training  is  completed  effectively 
and  accounts  for  all  the  requirements  of  an  off-site  training  location. 

c.  Training  of  Trainers 

It  is  a  false  assumption  that  just  because  someone  knows  how  to  do 
something,  then  they  can  just  as  well  teach  other  people  how  to  do  it.  There  are  two  possible 
approaches  to  selecting  and  training  the  actual  trainers; 

•  Train  personnel  with  good  communication  skills  to  use  the  software. 

•  Train  a  user-expert  to  train  others. 

The  attitude  and  approach  of  the  trainer  are  very  important.  If  the  trainer  is  unable  to  deliver 
the  material  to  the  trainees,  the  results  of  the  trmning  will  most  probably  be  ineffective.  The 
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use  of  an  outside  consultant  or  even  the  programmers  of  the  software  may  be  cost-prohibitive 
and  may  require  too  much  effort  in  the  “train  the  trainers”  area.  (Nelson,  1995) 
d.  Prelindnaty  Evaluation  of  Training  Methods  and  Materials 
Before  starting  the  actual  training,  an  evaluation  of  the  training  plan  must  be 
performed.  The  main  points  to  be  conadered  are  whether  the  training  is  providing  the  trainee 
with  the  desired  and  needed  skills,  whether  the  target  group  is  the  right  one,  and  whether  the 
methods  are  effective.  The  easiest  and  most  effective  way  to  do  this  is  to  forward  a  copy  of 
the  training  plan  to  the  deployment  team,  software  development  personnel,  and  users  for 
review.  The  goal  is  to  let  the  interested  parties  have  some  input  into  what  kind  of  training 
they  will  recdve.  After  the  responses  are  received,  necessary  changes  are  made  and  reviewed 
by  the  training  team  before  deployment.  (Leinonen,  1995) 

&  Formal  Training  and  Learning 

The  actual  training  will  be  conducted  according  to  the  training  plan,  but  not 
too  rigorously — if  new  problem  areas  emerge  during  the  actual  training,  they  should  be  dealt 
with  promptly.  Prop^  training  of  users  has  already  been  identified  as  a  critical  success  factor. 
If  not  carried  out  properly,  it  can  lead  to  costly  delays,  problems  and  dissatisfaction  on  the 
part  of  the  users  (Le  Compagnon  and  Leydon,  1991).  In  general,  the  training  should  be  a 
balance  between  knowledge-based  training  and  skills-based  training.  The  training  plan  should 
have  also  been  geared  to  the  specific  groups  that  need  to  learn  about  the  new  software.  With 
these  considerations  in  mind  while  formulating  the  training  plan,  it  is  anticipated  that  the 
major  training  pitfalls  of  inapplicable  training  material  and  ineffective  trainers  will  be  avoided. 
(Nelson,  1995) 


35 


f.  Evaluation  of  Training 

At  the  end  of  the  training  sessions,  an  evaluation  will  be  distributed  to  the 
trainees.  The  questions  in  the  survey  should  assess  the  following  areas:  (Cousins,  1994) 

•  Presentation  of  the  material. 

•  Skills  of  the  presenter. 

•  Whether  the  trainees  felt  the  objectives  of  the  training  were  met. 

•  Classroom  facilities. 

The  trainers  will  assure  the  trainees  that  the  evaluations  will  be  read  carefully  with  the  goal 
to  improve  the  training  for  future  participants.  For  training  to  be  effective,  it  must  be  seen 
as  an  ongoing  process  which  begins  with  a  determination  of  institutional  and  individual  needs, 
involves  both  users  and  trainers  in  the  planning  process,  and  includes  procedures  for 
evaluating  the  effectiveness  of  the  training  and  making  changes  and  adjustments  as  needed. 
The  training  evaluation  has  been  proven  to  be  an  effective  means  of  fine-tuning  individual 
training  programs.  It  is  also  a  means  of  determining  spedfic  areas  for  which  adequate  training 
is  not  being  provided.  However,  a  survey  will  not  provide  much  information  on  some  of  the 
questions  which  the  trainers  need  to  consider  if  training  is  to  be  a  part  of  the  deployment 
process.  (Le  Compagnon  and  Leydon,  1991)  Some  of  these  questions  on  the  survey  include: 

•  Does  the  training  make  the  user  more  self-sufScient? 

•  Does  it  make  the  user  more  eflScient  at  his  job? 

•  Does  it  provide  a  more  cost-effective  means  of  support  to  users  than  other  types 
of  support? 
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With  a  well  thought-out  training  plan  and  a  survey  that  queries  the  users  about  the  training, 
the  training  can  be  conducted  efficiently  and  productively  while  being  improved  in  the  areas 
that  trainees  specify  if  needed.  (Le  Compagnon  and  Leydon,  1991) 

3.  Conclusions  About  Training 

There  is  no  one  right  approach  to  effective  user  training  (Le  Compagnon  and  Leydon, 
1991).  Effective  training  can  take  on  any  number  of  different  forms,  not  all  of  which  take 
place  in  the  traditional  classroom  environment  with  an  instructor.  Documentation,  online 
tutorials,  and  video  tapes  can  provide  alternative,  adequate,  and  cost-effective  training  for 
most  user  needs.  These  alternatives,  however,  can  not  replace  the  responsiveness  of  an 
instructor  in  person.  The  training  must  also  be  seen  as  a  process  to  strengthen  long-term 
institutional  goals  and  as  a  cost-effective  means  of  providing  user  support  (Le  Compagnon 
and  Leydon,  1991).  Often,  not  enough  time  is  spent  up  front  assessing  needs  and  planning 
the  training  effort,  nor  is  enough  time  spent  evaluating  the  results  and  making  necessary 
changes.  With  prop©-  planning  and  evaluating,  training  can  be  one  of  the  most  cost-effective 
means  of  providing  computing  support  to  users. 

D.  EFFECTIVE  COMMUNICATIONS  BETWEEN  USERS  AND  DEVELOPERS/ 
MAINTENANCE  TEAM 

A  successful  implementation  does  not  end  with  the  deployment  of  the  software.  Once 
the  software  is  deployed  there  must  be  some  mechanism  for  the  users  to  communicate  with 
the  developers  and/or  the  maintenance  team.  Lines  of  communication  should  be  a  two-way 
street  that  provide  foUow-up  and  feedback.  The  users  must  have  a  point  of  contact  in  the 
event  that  they  have  a  question  or  complaint  about  the  software.  Information  should  also  be 
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available  on  project  objectives,  status,  changes,  organizational  coordination,  and  user  needs 
(Schultz,  Slevin,  and  Pinto,  1987).  Providing  a  point  of  contact  also  provides  an 
organizational  “home”  for  the  software.  The  need  for  assigning  a  system  manager  has  been 
clearly  demonstrated  (Keen  and  Morton,  1978).  The  system  manager  that  is  handling  the  user 
support  should  be:  (Keen  and  Morton,  1978) 

•  Familiar  with  the  job  of  the  users 

•  Skilled  technically,  administratively,  and  interpersonally 

•  Involved  in  planning,  priority  settiug,  and  decision  making  in  the  project 

An  effective  system  manager  also  facilitates  the  institutionalization  of  the  software  while  at 
the  same  time  encourages  and  makes  easier  user  participation  (Multinovich  and  Vlahovich, 
1984).  A  single  point  of  contact  also  avoids  conflict  by  having  one  person  prioritize  requests 
from  the  users. 

E.  post-implementahon  evaluation 

It  is  impossible  to  determine  implementation  success  without  a  thorough  evaluation 
(Cerullo,  1979).  The  determination  of  success  should  be  based  on  three  factors:  (Multinovich 
and  Vlahovich,  1984) 

•  A  prior  definition  of  “improvement”. 

•  A  means  of  monitoring  progress  toward  the  goals. 

•  A  review  process  to  determine  when  the  system  is  fully  implemented. 

In  the  case  of  a  newly  implemented  system,  performance  will  be  based  on  the  accuracy  of  the 
syst«n  resulting  from  the  user’s  use  of  the  system.  Measures  of  effectiveness  will  include  the 
realization  of  any  of  the  benefits  projected  from  the  cost-benefits  analysis.  Satisfaction  will 
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be  based  on  the  user’s  opinion  of  the  software.  A  survey,  an  example  of  which  is  included 
as  an  appendix  to  this  thesis,  should  be  distributed  to  measure  ease  of  use,  perceived 
accuracy,  and  the  perceived  impact  on  the  performance  of  the  user’s  duties  (Multinovich  and 
Vlahovich,  1984).  The  survey  should  be  designed  to  obtain  all  the  infonnation  needed  to 
judge  user  satisfaction  while  remaining  as  unobtrusive  to  the  user  as  possible.  But  how  can 
user  satisfaction  be  measured?  Lucas  and  Ginzberg  (1990)  have  formulated  a  model  for 
assessing  implementation  success  as  shown  in  Figure  3-1. 


Acceptance - ^Use - ►Performance - ►Satisfaction 


Figure  3-1.  Lucas-Ginzberg  model  of  implementation  success. 

The  model  shows  that  in  order  to  ensure  that  users  use  the  new  application,  they  must  be 
satisfied  with  it.  A  survey  can  gauge  user  satisfaction  and  help  determine  where  areas  of 
improvement  lie  (Lucas  and  Ginzberg,  1990). 

F.  HARDWARE  IMPLEMENTATION  ISSUES 

While  choosing  the  right  hardware  is  obviously  an  important  choice  in  a  successful 
implementation,  there  is  no  standard  hardware  configurations  that  are  defined  for  the  different 
categories  of  expert  systems  due  to  the  unique  requirements  that  every  system  has.  The 
starting  point  for  detamining  the  hardware  that  will  eventually  be  deployed  should  start  with 
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an  analysis  of  the  minimum  hardware  required  to  effectively  run  the  software  (McGaha, 
1994).  Factors  that  should  be  considered  prior  to  deployment  are: 

•  Portability:  Do  the  users  or  the  environment  require  a  portable  computer? 

•  Commercial-off-the-shelf  vs.  Custom  design:  Is  the  system  able  to  run  on  a 
commercially  available  computer  or  does  it  require  unique  hardware  or  an 
uncommon  configuration? 

•  Operating  environment:  Is  the  operating  environment  of  the  deployed  system  harsh 
enough  to  require  ruggedized  computers? 

•  Cost:  Is  affordability  a  concern? 

•  Expandibility:  Does  the  hardware  need  to  be  expandable  in  anticipation  of  adding 
new  features  or  upgrading  an  application? 

•  Security:  Does  the  application  require  a  secure  operating  environment? 
Hardware  should  be  chosen  after  the  development  of  a  full  prototype  version  is  completed 
at  which  time  it  is  easiest  to  determine  the  needs  of  the  system  (Prerau,  1990). 

G.  SUMMARY 

This  chapter  examined  implementation  issues  that  must  be  taken  into  account  before 
deploying  a  software  application.  Obtaining  management  support,  involving  users  in  the 
development  of  software,  training  users,  establishing  effective  communications,  and 
conducting  a  post-implementation  survey  are  important  issues  to  implementation  success. 
Finally,  the  type  of  hardware  that  the  software  will  be  deployed  on  must  take  into  account  the 
six  factors  listed  in  Section  J.  In  his  thesis,  Leonard  (1996)  lists  several  factors  that  are 
critical  to  implementation  success  based  on  published  research:  (Leonard,  1996) 
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•  Create  reaKstic  expectations  about  expert  system  capabilities. 

•  Involve  the  users  in  the  development  and  implementation  plan  to  the  degree  that 
they  feel  they  have  influence  in  the  process. 

•  Promote  system  usage  through  organizational  rewards. 

•  Provide  a  system  of  training  that  instills  confidence  in  the  -users  and  offers  the 
opportunity  for  additional  instruction  on  request. 

•  Build  enthusiastic  support  at  all  levels  of  management. 

•  Establish  effective  lines  of  communication  between  the  users  and  the  maintenance 
team  by  establishing  a  help  desk  and  points  of  contact  of  system  advocates. 

The  implementation  issues  examined  in  this  chapter  will  be  applied  in  Chapter  V  to  the 

implementation  of  the  MK  92  MAES  full  prototype  version. 
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IV.  IMPLEMENTATION  ISSUES  OF  MAES  AS  A  SOFTWARE 

SYSTEM 


The  previous  chapter  addressed  general  implementation  issues  involved  in  deploying 
a  software  application.  This  chapter  will  apply  the  implementation  issues' discussed  by  Prerau 
(1990)  spedfically  to  the  initial  deployment  of  MAES.  Section  A  provides  an  analysis  of  the 
general  software  implementation  issues  that  affect  MAES.  It  begins  with  a  description  of  the 
MAES  development  history  and  how  it  has  followed  the  fi'amework  developed  by  Prerau 
(1990).  Section  B  presents  lessons  learned  by  other  DOD  activities  that  implemented  expert 
systems  to  diagnose  weapons  systems.  Section  C  examines  organizational  structures  within 
the  Navy  and  thdr  impact  on  implementation.  Section  D  applies  research  on  management  and 
user  perspectives  to  the  implementation  of  MAES  and  Section  E  is  a  summary  of  the  chapter. 
A.  MAES  DEVELOPMENT  HISTORY 

The  general  software  development  life  cycle  model  used  by  the  MK  92  MAES 
development  team  generally  follows  a  model  described  by  Prerau  (1990).  He  divides  the 
expert  ^stem  life  cycle  into  three  phases:  the  initial  phase,  core  development  phase,  and  final 
development  and  deployment  phase  (Prerau,  1990).  A  summary  of  the  MAES  development 
for  these  phases  follows. 

1.  Initial  Phase 

The  initial  Prerau  phase  consists  of  project  start-up,  selection  of  the  domain,  and 
selection  of  the  development  environment.  In  1991,  engineers  at  NSWC-PHD  started  the 
devdopment  of  an  expert  system  designed  to  diagnose  problems  associated  with  the  MK  92 
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MOD  2  FCS  Daily  System  Operability  Test  (DSOT).  The  domain  expert  for  analyzing  and 
documenting  the  knowledge  to  be  incorporated  in  the  system  was  a  technical  representative 
from  UNISYS  with  over  25  years  engineering  experience.  The  knowledge  for  MAES  was 
acquired  from  MK  92  technical  manuals,  the  domain  expert’s  analysis  of  the  system,  and 
heuristics  of  other  MK  92  experienced  engineers. 

The  development  environment  chosen  for  MAES  was  an  expert  system  shell  from 
SoftSell  called  Adept™  v2.2.  This  tool  was  selected  after  extensive  evaluation  of  diagnostic 
expert  system  shells  and  the  experience  of  an  Army  development  team  building  an  ejq)ert 
system  for  the  Ml  A1  tank’s  turbine  engine.  Adept  was  selected  for  several  reasons.  First, 
Adept  uses  a  graphical,  procedural  approach  to  knowledge  representation  that  is  suitable  for 
diagnostic  systems.  Second,  it  has  an  integrated  graphical  user  interface  (GUI)  screen  builder. 
In  addition,  the  tool  is  easy  to  use  and  provides  for  unlimited  run-time  versions.  Finally,  the 
development  version  runs  on  Windows  based  80486  platforms  and  is  relatively  inexpensive 
at  $695.00  per  copy. 

2.  Core  Development  Phase 

The  core  development  phase  consists  of  the  development  of  a  feasibility  prototype  and 
a  full  prototype  system.  An  initial  prototype  of  MAES  was  buUt  and  demonstrated.  A  full 
prototype,  consisting  of  the  calibration  and  performance  modules,  was  then  developed  and 
fielded  to  the  USS  Sides  (FFG-14)  and  the  Fleet  Training  Center  in  San  Diego  (FTC-SD)for 
feedback.  The  prototype  deployment  allowed  senior  technicians  both  on  the  ship  and  at  the 
MK  92  technician  school  to  make  helpful  suggestions.  Examples  include: 
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•  The  use  of  photos  to  asast  in  following  recommended  troubleshooting/repair  steps 
and  for  which  pictures  are  not  available  in  the  tech  manuals. 

•  The  inclusion  of  “why”  buttons  to  explain  to  a  technician  why  a  certain  step  is 
recommended.  This  feature  also  enhances  the  training  capability  of  MAES. 

•  The  listing  ofpart  numbers  ^)^ilen  a  part  is  recommended  for  replacement.  This  will 
speed  the  supply  administrative  process  of  acquiring  a  new  part. 

Bradley  and  Hauser  point  out  that  when  technicians  observe  their  suggestions  being  turned 

into  program  features  it  gives  them  a  sense  of  accomplishment  and  involvement  in  a  project 

(Bradley  and  Hauser,  1995).  All  recommendations  were  incorporated  into  the  system. 

Involving  the  instructors  at  FTC-SD  also  served  the  purpose  of  introducing  MAES  to  student 

technicians  that  were  about  to  enter  the  fleet.  This  was  a  method  to  ejqpose  potential  users 

to  MAES  prior  to  reaching  thdr  ships.  Student  technidans  at  the  MK  92  “C”  school  also  had 

the  advantage  of  receiving  the  introductory  MAES  training  over  the  course  of  a  week  instead 

of  the  one  day  training  schedule  planned  on  for  the  initial  deployment. 

3.  Final  Development  and  Deployment  Phase 

The  MAES  project  is  currently  in  this  phase.  During  the  development  and  deployment 
phase,  a  production  system  is  built  and  the  system  is  deployed.  MAES  will  then  transgress 
to  the  maintenance  phase,  with  software  and  knowledge  bugs  being  fixed  and  new  information 
and  features  being  added  as  necessary.  To  produce  the  initial  deployment  version  of  MAES, 
several  issues  were  examined. 

•  Possible  representation  and  reimplementation:  Although  a  prototype  of  the 
performance  module  had  been  completed,  an  outside  contractor  was  hired  to 
redesign  the  software,  incorporate  new  knowledge  into  the  module  and  ensure  that 
the  module  was  robust  and  reliable  for  fielding. 
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•  Deployment  Hardware:  Factors  that  were  considered  for  MAES  hardware 
dq)loyment  were  cost,  reliability,  portability,  and  screen  characteristics.  This  topic 
is  discussed  in  greater  detail  in  Chapter  V. 

•  Deployment  software:  The  deployment  software  is  simply  a  run-time  version  of  the 
development  software.  Users  will  be  able  to  run  MAES  without  having  to  install 
the  development  software,  Softsell’s  Adept  2.2™,  on  their  computer.  Users  will 
not  be  able  to  make  changes  to  the  distributed  software. 

•  Input/Output  formats:  Comments  from  users  have  reflected  that  the  user  interface 
is  extremely  easy  to  learn  and  use.  In  recent  tests,  non-MK  92  technicians  were 
able  to  diagnose  MK  92  faults  correctly  using  MAES,  even  though  they  had  no 
prior  exposure  or  experience  with  the  MK  92  FCS  (Myer,  1996). 

•  Testing:  The  initial  version  of  the  calibration  module  has  been  tested  and 
evaluated.  FTC-SD  has  been  verifying  knowledge  by  allowing  student-technicians 
to  use  MAES  to  diagnose  faults  on  a  MK  92  mock-up.  Knowledge  and  software 
validation  and  verification  have  been  part  of  the  development  process. 

•  Documentation:  NSWC  PHD  will  be  maintaining  MAES  after  the  deployment  of 
the  initial  version.  The  following  documentation  to  support  maintenance  is  being 
assembled  for  transfer  to  NSWC  PHD:  System  Level  Description,  Software 
Design  Document,  MAES  User’s  Manual,  Version  Description  Document,  and  a 
Program  Package.  Final  versions  will  be  produced  as  part  of  the  production 
decision  to  deploy  to  all  ships. 

The  implementation  plan  for  the  deployment  of  the  initial  version  provides  a  guideline  for 
completing  the  steps  required  for  deployment  of  MAES  and  is  included  as  Appendix  A  to  this 
thesis. 


The  long  term  maintenance  of  MAES  will  be  conducted  by  NSWC-PHD.  Having 
NSWC-PHD  as  the  only  maintenance  source  allows  a  centralized  approach  to  maintenance 
(Prerau,  1990).  Work  has  begun  on  defining  and  preparing  the  deliverables  required  to 
support  the  fielded  system. 


46 


B.  INCORPORATING  LESSONS  LEARNED  THAT  MAY  IMPACT 

IMPLEMENTATION 

This  section  addresses  the  subadiary  research  question  “What  are  the  lessons  learned 
in  the  implementation  of  other  ejq)ert  systems  for  weapons  systems  within  DOD?”  Although 
personal  computers  and  software  applications  are  widely  used  throughout  DOD,  expert 
system  software  is  a  relatively  scarce  technology  that  is  still  not  common  within  the  military. 
However,  two  large  expert  systems  have  recently  been  deployed  by  the  Army  and  Navy: 

•  The  Turbine  Engine  Diagnostic  (TED)  expert  system  is  used  in  the  Army 
intermediate  maintenance  activities  to  aid  gas-turbine  technidans  in 
troubleshooting  the  turbine  engines  installed  in  the  Ml  Abrams  main  battle  tank. 

•  The  Phalanx  Integrated  Maintenance  System  (PIMS)  used  in  the  Navy  to  aid  fire 
control  technicians  on  surface  ships  in  maintaining  and  diagnosing  faults  in  the 
Phalanx  close-in  weapon  system  (CIWS)  was  deployed  over  the  past  two  years. 

Valuable  lessons  learned  from  the  deployment  of  these  two  systems  can  be  utilized  to  avoid 

problems  in  the  MK  92  MAES  deployment.  The  lessons  learned  from  these  systems  are 

espedally  applicable  to  this  deployment  because: 

•  Both  expert  systems  were  deployed  for  military  systems. 

•  Both  expert  systems  have  requirements  similar  to  the  MK  92  MAES  in  that  they 
were  both  designed  to  aid  technidans  in  the  maintenance  of  complex,  technical 
machinery. 

•  Both  expert  systems  are  used  primarily  by  enlisted  technicians. 

Video  teleconferences  (VTC)  with  the  Army  Research  Laboratory  (ARL)  at  Aberdeen 
Proving  Grounds  (which  developed  and  deployed  TED)  and  the  Naval  Ordnance  Station 
(NAVORDSTA)  at  Louisville,  KY,  (which  developed  and  deployed  PIMS)  were  held.  Many 
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valuable  lessons  were  learned  in  these  teleconferences.  They  fall  into  several  categories 
defined  below. 

•  Training:  Lessons  learned  that  improve  training  methods  and  matftriak  during 
initial  implementation. 

•  Maintenance:  Lessons  learned  that  improve  the  post-deployment  maintenance  of 
the  ejq)ert  system. 

•  Post-implementation  evaluation:  Lessons  learned  that  improve  the  conduct  and 
effectiveness  of  the  post-implementation  survey. 

•  Ebrdware  issues:  Lessons  learned  that  may  aid  in  making  choices  in  which  type  of 
hardware  to  deploy  MAES. 

The  following  sections  discuss  the  lessons  learned  in  more  detail. 

1.  Training  Lessons  Learned 

The  training  lessons  learned  include: 

•  Training  needs  to  allow  for  a  wide  range  of  computer  experience.  Trainers  should 
be  prepared  to  deal  with  computer  illiterate  personnel. 

•  Training  teams  need  to  visit  each  site  to  conduct  the  initial  training.  Besides  the 
obvious  benefits  to  the  new  user,  the  developers  learned  much.  Trainers  were  able 
to  stand  beside  the  technicians  as  they  observed  how  the  user  used  the  expert 
system  and  saw  how  they  reacted.  As  a  result  they  were  able  to  gain  first  hand 
knowledge  on  where  the  deficiencies  were  in  the  user  interface. 

•  Users  need  to  be  informed  that  the  issued  computer  is  only  to  be  used  for 
accessing  the  expert  system.  Maintenance  teams  received  many  calls  about 
software  conflicts  resulting  fi-om  the  installation  of  unauthorized  software  and 
there  is  a  real  threat  of  the  introduction  of  viruses. 

•  Training  needs  to  be  coordinated  with  the  commands  receiving  the  expert  system 
as  far  ahead  of  time  as  possible.  A  lack  of  coordination  between  activities,  will 
likely  result  in  dfficulty  getting  the  right  people  to  attend  training  sessions.  ’ 

•  Deployment  teams  should  not  install  multimedia  training  aids  on  computers.  A 
training  video  installed  on  laptop  computers  was  not  found  to  be  worth  the  time, 
money,  and  hard  drive  resources  required. 
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2.  Maintenance  Lessons  Learned 


The  maintenance  lessons  learned  include: 

•  The  deployment  team  should  solidt  users  for  suggestions  during  training  sessions. 
Major  revisions  to  the  software  were  made  after  deployment  based  on  user 
feedback. 

•  Developers  should  be  sensitive  to  the  varying  levels  of  computer  expertise  that 
enlisted  technicians  may  have.  Expect  to  add  more  on-line  help  for  the 
inexperienced  user. 

•  A  single  point  of  contact  should  be  designated  for  problems  with  the  software.  A 
single  point  of  contact  provides  consistency  to  the  users  that  have  problems  with 
the  software. 

•  Technical  support  personnel  must  be  prepared  to  deal  with  general  computer 
problems.  A  large  number  of  requests  for  assistance  were  for  problems 
unassociated  with  the  expert  system,  such  as  computers  that  would  not  boot  or 
problems  with  the  Windows  operating  ^stem.  Plan  ahead  and  prepare  for  these. 

3.  Post-Implementation  Evaluation  Lessons  Learned 

The  post-implementation  evaluation  lessons  learned  include; 

•  The  maintenance  team  should  have  in  place  a  web  page  for  technical  support. 
Extensive  use  of  the  Internet  was  made  to  make  revisions  available  for  download 
and  to  provide  a  method  for  technicians  to  provide  feedback  and  bug  reports. 

•  The  maintenance  team  should  include  a  survey  that  can  be  answered  by  individual 
users  on  a  web  page.  It  was  found  that  technicians  were  not  answering  surveys 
that  were  mailed  out.  Once  world  wide  web  access  was  available,  the  problem  of 
collecting  data  diminished. 

4.  Hardware  Lessons  Learned 

The  hardware  lessons  learned  include: 

•  All  technicians  should  have  access  to  the  computer  that  contains  the  expert  system. 
Some  senior  technicians  kept  the  computers  locked  up  for  fear  of  pilferage, 
limiting  access  to  others. 
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•  Training  should  emphasize  that  this  system  is  for  troubleshooting  purposes  only. 
Some  senior  technicians  used  the  computers  for  other  purposes,  such  as 
administrative  tasks. 

•  Ruggedized  laptops  are  not  necessary.  Offtheshelflaptops  have  so  far  fared  well 
in  an  intermediate  maintenance  garage  type  environment. 

•  The  computers  that  are  distributed  should  have  durable  pointing  devices. 
Trackball  and  mouse-type  pointing  devices  tend  to  become  inoperative  in  a 
maintenance  environment. 

•  At  a  minimum,  dual  scan  screen  technology  should  be  used  for  laptop  computers. 
If  the  laptop  will  be  used  in  bright  sunlight,  then  an  active  matrix  screen  should  be 
specified.  Technicians  had  difficulty  discerning  details  in  photos  when  black  and 
white  displays  were  used. 

C.  ORGANIZATIONAL  STRUCTURES  AND  THEIR  IMPACT  ON 
IMPLEMENTATION 

Lucas  and  Ginzberg  (1990)  developed  an  implementation  model  that  addresses 
management  and  user  concerns  in  the  implementation  process  Before  the  factors  in  the  sub¬ 
models  can  be  examined,  the  organizational  structure  of  the  ship  and  the  shore-based 
maintenance  and  training  fedMes  must  be  considered.  A  determination  of  who  fits  the  roles 
of  management  and  user  must  first  be  resolved. 
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1.  Shipboard  Organizational  Structure 

An  organizational  chart  of  the  typical  chain  of  command  onboard  a  FFG-7  is  shown 
below  in  Figure  4-1 . 


Figure  4-1.  FFG-7  Chain  of  Command. 


System  operation  and  maintenance  aboard  ship  are  conducted  by  enlisted  technicians  who 
possess  the  Fire  Control  (FC)  rating  with  a  Navy  Enlisted  Classification  (NEC)  of  1 102.  The 
technidans  receive  this  NEC  once  they  have  completed  the  32  week  MK  92  “C”  school.  The 
manning  level  for  MK  92  FCs  on  FFG-7  class  ships  is  seven,  but  may  vary  fi'om  ship  to  ship. 
Reserve  ships  are  allotted  one  less  junior  technician.  Budget  cuts  and  downsizing  have 
contributed  to  the  lack  of  proper  manning  levels  in  the  fleet.  Reduced  manning  is  a 
contributing  factor  for  the  high  number  of  technical  assistance  requests.  (McGaha,  1994) 
Holek  (1994)  describes  three  types  of  personnel  that  need  to  be  trained:  upper 
management,  line  management,  and  users.  Applying  this  framework  to  a  shipboard 
organization  would  result  in  the  following  classifications: 
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•  Upper  management:  The  commanding  officer,  executive  officer,  and  combat 
systems  officer. 

•  Line  management:  The  division  officer  and  division  chief. 

•  Users:  All  MK  92  technicians;  junior  and  senior. 

Classifying  shipboard  personnel  in  the  above  manner  will  allow  the  MAES  deployment  team 
to  tailor  their  training  plan  to  each  category  of  manager  or  user. 

2.  Shore-based  Maintenance  and  Training  Organizations 

Unlike  the  shipboard  organization,  there  is  no  hierarchy  associated  with  the  shore- 
based  maintenance  and  training  commands.  After  enlisted  recruits  graduate  from  boot  camp 
and  “A”  school,  they  are  sent  to  the  “C”  school  for  which  they  have  qualified.  In  the  case  of 
MK  92  technicians,  the  “C”  school  is  located  at  FTC-SD  and  FCTC  Dam  Neck.  Although 
the  schools  are  not  a  part  of  the  maintenance  organization,  they  have  proved  invaluable  as  a 
source  for  testing  the  knowledge  contained  within  MAES.  The  school  has  also  served  as  a 
way  to  expose  new  MK  92  technidans  to  MAES  in  a  training  environment  and  to  gather  data 
on  the  system. 

Technical  support  is  available  to  shipboard  technicians  from  Fleet  Technical  Support 
Centers  (FTSCs)  and  NSWC-PHD.  A  ship  requests  technical  assistance  through  the 
respective  type  commander’s  (TYCOM)  Readiness  Support  Group  (RSG)  via  telephone  or 
CASREP.  FTSC’s  are  staffed  by  both  senior  Navy  enlisted  technicians  and  experienced 
civilian  technicians.  FTSC  technical  representatives  can  be  considered  line  mflnagfimftnt  due 
to  their  position  as  technical  experts.  However,  there  are  different  levels  of  experience  and 
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expertise  at  the  FTSC.  Some  technidans  may,  on  occasion,  use  MAES  in  areas  they  are  less 
knowledgeable  about.  They  would  be  considered  users  in  such  cases. 

D.  MANAGEMENT  AND  USER  PERSPECTIVES  ON  MAES 

IMPLEMENTATION 

The  Lucas-Ginzberg  (1990)  model  of  implementation  consists  of  two  stages:  adoption 
of  a  system  by  the  managers  and  adoption  of  the  ^stem  by  users.  This  model  is  applicable 
to  the  FFG-7  shipboard  organization.  The  following  sections  discuss  these  two  areas. 

1.  Management’s  Perspective  of  MAES  Implementation 

The  importance  of  management  support  and  its  contribution  to  implementation 
success  was  discussed  in  detail  in  Chapter  El.  Although  the  two  sub-models  from  the  Lucas- 
Ginzberg  (1990)  implementation  model  include  many  factors  that  influence  implementation 
success,  only  the  specific  factors  that  can  be  influenced  by  the  MAES  development  and 
deployment  teams  are  addressed  here. 

•  Manager  acceptance:  On  a  US  Navy  surface  combatant,  enlisted  technicians  are 
influenced  in  their  work  habits  most  strongly  by  the  division  chief  and  somewhat 
by  the  division  ofiScer.  Manager  acceptance  will  in  turn  be  influenced  by  manager 
knowledge  of  MAES  and  manager  assessment  of  the  MAES  system  and  available 
support. 

•  Manager  knowledge  of  MAES:  The  MAES  deployment  team  will  instruct  division 
officers  and  chiefe  on  the  benefits  and  purpose  of  MAES  and  the  basics  of  how  the 
system  woiks.  Acceptance  of  MAES  should  increase  with  management  education. 

•  Manager  assessment  of  MAES  and  support:  Showing  officers  and  chiefr  the 
advantages  and  benefits  that  MAES  equipped  technicians  will  have  over 
technicians  not  using  MAES  should  gain  their  support  for  the  system.  Providing 
the  division  officer  and  chief  with  a  means  of  repairing  hardware  faults  and 
providing  technical  support  for  MAES  software  will  gain  their  confidence  of  the 
system. 
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•  Manage  belief  in  the  MAES  concept:  OflScers  and  chiefs  who  believe  in  MAES  ’ 
ability  to  aid  technidans,  cut  down  the  NFE  problem,  and  reduce  MTTR  will  want 
to  learn  about  the  ^stem.  With  this  knowledge  they  will  encourage  the  technicians 
to  use  it. 

•  Top  management  support;  The  deployment  team  should  address  the  commanding 
officer,  executive  officer,  and  the  combat  systems  officer  during  an  informal,  half- 
hour  brief  on  MAES.  This  would  provide  an  opportunity  to  inform  the  ship’s 
upper  management  of  MAES  benefits.  A  short  demo  should  be  included.  This  is 
an  important  brief  as  the  ship’s  upper  management,  if  they  believe  in  the  system’s 
importance,  can  greatly  influence  how  the  division  officer  and  chief  view  MAES. 

Obtaining  upper  management’s  support  can  significantly  increase  the  probability  of  a 

successful  implementation  of  the  system  (Lucas-Ginzberg,  1990). 

2.  User’s  Perspective  of  MAES  Implementation 

Improving  the  technidan’s  ability  to  correctly  diagnose  and  solve  faults  in  the  MK  92 
MOD  2  PCS  is  one  of  the  primary  goals  of  MAES.  A  short  discussion  of  each  of  the 
applicable  user  variables  in  the  Lucas-Ginzberg  model  that  can  be  influenced  by  the 
implementation  and  how  they  may  be  applied  to  MAES  follows. 

•  User  acceptance;  This  is  a  key  variable  on  whether  or  not  a  system  will  be  used. 
It  is  influenced  by  the  user  variables  discussed  below.  The  instruction,  demos,  and 
hands-on  training  should  all  be  oriented  at  achieving  user  acceptance. 

•  User  knowledge  of  MAES:  For  users  to  accept  a  system,  they  must  be 
knowledgeable  of  it.  The  MAES  deployment  team  will  train  the  technicians  on 
how  MAES  functions,  its  capabilities,  and  its  limitations. 

•  User  assessment  of  MAES  and  support:  Training  of  the  users  must  ensure  that 
they  have  the  necessary  means  to  resolve  hardware  and  software  problems.  A 
single  point  of  contact  for  providing  MAES  support  should  be  established. 
NSWC-PHD,  as  system  configuration  manager,  will  be  so  designated. 

•  MAES  characteristics:  This  variable  represents  the  features  and  capabilities  of 
MAES.  Easy  to  use  systems  such  as  MAES  are  more  likely  to  be  accepted  by  the 
user.  The  implementation  process  should  cover  this  variUble. 
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•  User-researcher  involvement:  Although  it  was  not  possible  for  every  technician  to 
be  involved  in  the  MAES  development,  users  should  be  made  aware  of  the  fact 
that  some  of  the  best  features  of  MAES  have  resulted  from  sailor’s  suggestions 
during  shipboard  prototype  demonstrations  and  evaluations  at  the  training  centers. 
This  should  provide  a  sense  of  user  ownership  of  the  system. 

•  Us©"  perception  of  management  support:  The  division  officer  and  chief  must  foster 
an  environment  of  support  through  their  encouragement  of  MAES  use  when 
diagnosing  applicable  MK  92  casualties.  Users  should  also  be  aware  of  upper 
management’s  support  for  MAES.  Encouraging  MAES  use  as  a  training  tool  to 
study  for  rating  exams  and  as  a  tool  for  conducting  training  sessions  will  aid  user 
perception  of  management  support. 

•  User  knowledge  of  system  purpose  and  use:  Training  in  this  area  will  be 
concurrent  with  the  variable  “user  knowledge  of  MAES”.  The  difference  is  the 
focus  on  the  purpose  of  MAES.  Technicians  need  to  be  made  aware  of  the  high 
cost  of  NFEs,  the  high  MTTR,  and  the  increasing  reduction  in  tech  assist  support. 

•  Problem  urgency:  This  variable  reflects  the  criticalness  and  urgency  of  the 
problems  (NFE’s,  high  MTTR,  reduction  in  tech  support)  to  which  MAES 
addresses.  Educating  the  technicians  on  the  benefits  of  saving  them  time  and 
saving  their  ship  money  increases  their  stake  in  MAES’  success. 

•  Organizational  support:  This  is  a  measure  of  the  degree  to  which  the  deployment 
team  can  convince  management  to  allow  technicians  the  time  to  train  on  MAES. 

Most  of  the  above  variables  can  be  positively  influenced  by  how  effectively  the  MAES 

deployment  team  conducts  the  training  during  their  deployment  visits  to  the  FFGs.  A  well 

thought-out  and  organized  training  plan  that  takes  these  variables  into  consideration  will  be 

a  crucial  factor  in  the  implementation’s  success. 

E.  SUMMARY 

Although  MAES  is  an  expert  system,  general  software  implementation  issues  should 
not  be  ignored.  Adhering  to  an  established  and  tested  development  methodology,  such  as 
Prerau’s,  provides  a  foundation  for  a  successful  implementation.  Also,  the  experience  that 
other  organizations  have  in  implementing  a  similar  system  can  be  invaluable.  If  problem  areas 
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were  encountered  in  prior  e^ert  ^stem  implementations,  they  can  be  avoided  by  determining 
what  lessons  have  been  learned. 

To  properly  address  each  level  of  an  organization  that  may  use  MAES,  the 
organizational  structure  and  how  it  impacts  an  implementation  must  be  understood.  Then, 
an  implementation  may  be  developed  that  addresses  the  concerns  of  both  the  managers  and 
the  users.  The  next  chapter  addresses  implementation  factors  that  are  specific  to  the  MAES 
deployment. 
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V.  IMPLEMENTATION  ISSUES  FOR  THE  INITIAL 
DEPLOYMENT  OF  MAES  AS  AN  EXPERT  SYSTEM 


This  chapter  applies  the  implementation  methodology  and  issues  discussed  in  the  first 
four  chapters  of  this  thesis  to  the  implementation  of  the  initial  version  of  the  MK  92  MAES. 
Section  A  classifies  MAES  in  accordance  with  the  Meyer  and  Curley  (1991)  scheme  and 
applies  expert  system  implementation  fectors  specifically  to  MAES.  Sections  B  and  C  discuss 
obtaining  chain  of  command  support  for  MAES  and  involving  MK  92  technicians  in  the 
implementation,  respectively.  Sections  D  and  E  recommend  approaches  to  training  and 
supporting  MAES  users.  Section  F  explains  how  to  conduct  and  evaluate  a  post¬ 
implementation  survey  while  Section  G  addresses  hardware  implementation  issues.  Section 
H  is  a  summary  of  the  chapter. 

A.  EXPERT  SYSTEM  IMPLEMENTATION  FACTORS 

As  previously  discussed,  ^ert  systems  possess  unique  characteristics  that  distinguish 
them  fi-om  traditional  management  information  systems.  The  fact  that  expert  systems  also 
vary  from  one  another  also  needs  to  be  taken  into  consideration.  The  following  sections 
discuss  how  MAES  is  classified  as  an  expert  system  and  additional  considerations  that  need 
to  be  made  for  implementing  MAES. 

1.  OassifyingMAES 

Meyer  and  Curley  (1991)  developed  a  classification  scheme  that  views  expert  systems 
along  two  attributes:  knowledge  complexity  and  technology  complexity.  MAES  best  fits  into 
the  cat^ory  “Knowledge  Intensive”,  since  it  incorporates  the  knowledge  of  skilled  decision 
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make's,  requires  a  simple  computing  environment,  and  aids  the  decision  making  for  a  specific 
group.  MAES  has  the  following  characteristics: 

•  MAES  incorporates  the  knowledge  of  skilled  and  ejqperienced  engineers. 

•  MAES  requires  a  simple  computing  environment.  MAES  is  a  point-and-click 
^stem  that  runs  efBciently  on  a  486-DX4  75  MHz  laptop  with  8  Mb  of  RAM. 

•  MAES  aids  decision  making  for  a  specific  group.  This  group  includes  enlisted 
technicians  and  technical  representatives  responsible  for  maintaining  the  MK  92 
MOD2FCS. 

•  MAES  was  developed  using  an  expert  system  shell. 

•  MAES  contains  complex  knowledge  and  represents  true  ejqjertise,  instead  of 
commonly  used  procedures. 

These  characteristics  therefore  most  closely  are  associated  with  the  “ICnowledge  Intensive” 
category.  Knowing  which  category  of  expert  system  that  MAES  falls  under  aids  in 
implementation  planning  since  each  type  of  expert  ^stem  is  affected  differently  by 
implementation  factors  (Bradley  and  Hauser,  1995). 

2.  Expert  System  Factors  That  Affect  MAES  Implementation 

The  feet  that  MAES  is  an  expert  system  carries  with  it  additional  considerations  for 
implementation.  These  were  addressed  in  general  in  Chapter  HI.  Figure  5-1  is  a  summary 
of  how  Bradley  and  Hauser  (1995)  rate  the  relative  importance  of  implementation  factors  for 
different  e3q)ert  systems.  MAES’s  category  is  encircled.  Knowing  which  factors  will  be  most 
influential  in  the  implementation  and  applying  them  increases  the  probability  of  a  successful 
implementation  (Bradley  and  Hauser,  1995).  Also,  since  MAES  is  a  resource  constrained 
project  (both  in  funding  and  persoimel),  every  fector  may  not  be  able  to  be  fuUy  addressed. 


58 


Determining  where  efforts  should  be  concentrated  during  the  implementation  process  will 
reduce  the  risk  of  failure. 


Expert  System  Classification 
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L  ^  low  emphasis  in  the  implementation  plan 

medium  emphasis  in  the  implementation  plan 
H  =  high  emphasis  in  the  implementation  plan 

Figure  5-1.  Relative  Importance  of  Expert  System  Implementation 
Factors  with  MAES  Category  Highlighted.  (Bradley  and  Hauser,  1995) 


From  Figure  5-1,  the  factors  that  are  most  likely  to  have  a  major  influence  in  the 
implementation  of  MAES  are  as  follows: 

•  User  perception  of  computing  technology:  Greatly  dependent  on  the  complexity 
of  the  expert  ^stem.  The  uncomplicated  graphical  interface  that  MAES  uses  eases 
the  risk  of  technicians  becoming  overwhelmed  by  MAES.  A  thorough  tutorial  on 
how  to  use  MAES  will  be  an  integral  part  of  the  training  session  and  user’s 
manual.  All  users  should  receive  the  training  as  there  is  always  the  possibility  of 
a  technidan  being  computer  adverse  because  of  a  lack  of  familiarity  and  training. 

•  User  involvement  in  MAES  design  and  implementation:  Discussed  in  the  previous 
section,  users  have  provided  valuable  input  on  the  features  of  MAES.  This  fact 
needs  to  be  emphasized  during  implementation. 

•  Supervisor  support:  Analogous  to  management  support,  discussed  previously.  All 
levels  of  management  need  to  participate  in  the  implementation. 
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•  User  perception  of  change:  This  factor  is  associated  with  the  technicians 
perception  of  how  MAES  wiD  impact  their  work  routine.  Emphasizing  that  MAES 
is  just  another  piece  of  test  equipment  (albeit  a  much  more  effective  one)  should 
minimize  technician  perception  of  change. 

During  the  MAES  implementation,  emphasis  should  be  placed  on  the  technician’s  perception 
of  the  technology,  how  easy  it  is  to  use,  involvement  of  the  users  in  the  implementation,  and 
gaining  management  support. 

B.  OBTAINING  CHAIN  OF  COMMAND  SUPPORT  FOR  MAES 

As  discussed  in  Chapter  HI,  management  support  will  be  a  requirement  for  the 
successfiil  deployment  of  MAES.  Applying  methods  of  gaining  the  support  of  the  chain  of 
command  and  overcoming  resistance  to  MAES  are  discussed  in  this  section. 

Gainmg  the  support  of  the  chain  of  command  on  the  ship  and  of  senior  technical 
representatives  ashore  should  be  the  key  objective  for  the  MAES  deployment  team.  The 
introduction  of  a  new  technology  that  introduces  change  into  the  way  maintenance  is 
performed  will  meet  with  some  d^ee  of  resistance.  As  developers,  one  of  our  key  concerns 
is  that  MAES  be  used  when  appropriate.  If  it  is  not  used,  a  fair  evaluation  of  MAES  can 
never  be  made.  There  are  several  things  that  may  be  done. 

1.  Selling  MAES  to  the  Chain  of  Command 

Gaining  support  of  the  chain  of  command  is  the  primary  reason  that  the  deployment 
team  will  be  visiting  each  ship  the  day  after  training  the  MK  92  technicians.  A  short  brief  to 
the  commanding  ofiBcer,  executive  officer,  and  combat  ^sterns  officer  will  inform  them  of  the 
benefits  MAES  will  provide.  The  benefits  that  are  most  likely  to  appeal  to  them,  and  should 
be  stressed  during  the  brie^  are  MAES’  abffity  to  save  the  ship  money  (through  the  reduction 
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of  NFEs),  increased  operational  readiness,  and  improvement  in  the  sailor’s  quality  of  life 
through  a  reduction  in  the  long  hours  spent  troubleshooting  faults.  The  senior  chain  of 
command  must  be  made  aware  and  convinced  that  their  support  will  be  instrumental  in 
convindng  the  technicians  to  use  MAES.  Their  position,  in  the  form  of  a  directive  which  is 
either  oral  or  written,  that  MAES  shall  be  used  for  all  applicable  faults  can  have  a  tremendous 
influence.  Occasional  verbal  inquiries  on  the  system,  demonstrating  their  continued  interest, 
can  also  serve  as  a  strong  motivator. 

2.  Getting  Line  Management  Involved 

The  division  officer  and  chiei^  the  next  two  players  in  the  MK  92  chain  of  command, 
are  also  instrumental  to  a  successful  implementation.  They  have  daily  contact  that  is  a  direct 
influence  over  the  MK  92  technicians.  The  division  officer  and  chief  may  show  support  by: 

•  Attending  meetings  with  the  technicians  about  MAES. 

•  Ensuring  MAES  training  is  part  of  the  technician’s  training  plan  and  providing  time 
for  them  to  train  on  MAES.  General  Military  Training  (GMT)  is  usually 
conducted  at  least  three  times  a  week  onboard  a  ship.  Providing  time  for 
technicians  to  train  on  MAES,  as  little  as  once  a  month,  will  show  them  that  the 
chain  of  command  is  serious  about  the  importance  of  MAES. 

•  Becoming  knowledgeable  about  the  basic  workings  of  MAES.  It  is  a  practice  of 
good  leadership  for  officers  to  know  the  basics  of  the  equipment  for  which  they  are 
responsible.  Being  familiar  with  MAES  will  also  allow  the  division  officer  to 
better  complete  the  deployment  evaluation  that  is  critical  to  fleet  wide  deployment. 

•  Personally  using  MAES.  If  the  division  officer  or  chief  use  MAES,  even  if  just 
once  or  twice,  they  will  become  more  familiar  with  its  capabilities  and  limitations. 
This  understanding  of  the  system  should  provide  them  knowledge  to  support  then- 
insistence  that  MAES  be  used. 

Although  the  demands  on  an  officer  or  chiefs  time  are  many,  it  should  only  take  about  15 
minutes  to  learn  how  to  use  MAES  (Myer,  1996).  If  the  chain  of  command  is  able  to  devote 
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just  a  fraction  of  their  time  to  demonstrate  their  support  for  MAES,  they  play  a  key  role  in 
seeing  that  MAES  is  utilized. 

3.  Generating  Support  Through  Demonstration 

Conducting  a  brief  demonstration  of  the  system  to  the  chain  of  command  would  be 
an  effective  way  to  present  the  capabilities  and  advantages  of  MAES  (Bouldin,  1989).  The 
demonstration  must  be  simple  enough  to  be  brief  and  comprehensible.  It  should  focus  on  the 
key  features  of  the  system  and  MAES’  ease  of  use.  Illustrating  one  example  of  a  MAES 
troubleshooting  path  and  demonstrating  its  overall  capabilities  should  be  suflBcient.  The 
MAES  demonstration  team  should  keep  it  simple,  brief,  and  familiar  (Bouldin,  1989). 

4.  Gaining  Support  Through  Respected  Advocates 

Gaining  the  support  of  MK  92  technical  representatives  that  are  respected  in  their  field 
and  trusted  by  the  chain  of  command,  will  aid  in  gaining  support  for  MAES  (Bouldin,  1989). 
One  strategy  would  be  to  have  a  respected  MAES  advocate  available  at  the  MAES 
demonstration  onboard  the  FFGs.  The  key  activities  that  could  provide  the  respected 
advocates  include  the  Fleet  Technical  Support  Centers,  or  the  ISEA  at  NSWC-PHD.  The 
training  centers  may  also  be  a  source.  An  advocate  with  a  lot  of  experience  and  that  is  known 
to  technicians  onboard  the  ship  can  provide  tremendous  support  for  MAES  that  will  be 
passed  on. 

5.  Distributing  a  MAES  Newsletter 

Distributing  a  MAES  newsletter  to  each  ship  involved  in  the  implementation  would 
keep  management  and  technicians  involved  in  the  project’s  progress  and  provide  a  forum  for 
discussion.  The  newsletter  would  also  provide  a  medium  to  publicize  the  contributions  that 
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fleet  technicians  have  about  MAES  and  provide  suggestions  on  how  to  use  MAES  effectively. 
A  newsletter  could  be  published  as  infrequently  as  twice  a  year  and  be  small  enough  to  fill 
only  a  few  pages  and  still  serve  as  a  tool  for  developing  and  retaining  an  ongoing  support  by 
the  chain  of  command.  The  challenge  will  be  finding  the  resources,  both  funding  and 
personnel,  to  produce  such  a  newsletter. 

6.  Presenting  Information  from  the  MAES  Cost-Benefit  Analysis 

Multinovich  and  Phatak  (1981)  state  that  one  of  the  best  ways  to  obtain  management 
support  for  a  new  technology  implementation  is  by  presenting  them  the  results  of  a  favorable 
cost-benefit  analy^.  Cost-benefit  analysis  results  may  provide  a  means  that  is  easier  for  the 
senior  chain  of  command  to  imderstand.  Fortunately,  a  cost-benefit  analysis  for  MAES  was 
completed  in  September  1993  by  LCDR  Steven  Powell,  a  graduate  student  at  NPS.  Key 
results  that  should  be  presented  at  the  shipboard  briefing  are:  (Powell,  1993) 

•  Approximately  22%  of  MK  92  parts  turned  in  for  repair  were  No  Fault  Evident 
(NFE)  parts  and  CASREPs  from  ships  requesting  technical  assistance  required  an 
average  of  251  hours  until  repair. 

•  The  NFE  problem  is  costing  the  fleet  $900,000  in  OPTAR  funding  per  year  for  28 
Mod  2  ships.  Note:  the  original  cost  benefit  analysis  was  based  on  51  FFG-7s. 
These  figures  represent  current  CNO  plans  of  28  MK  92  MOD  2  deployed 
systems. 

•  With  the  Calibration  and  Performance  modules,  MAES  can  improve  operational 
readiness  8%  and  the  mean  time  to  repair  should  improve  by  1 5%. 

•  The  two  modules  in  version  1.0  of  MAES  can  cover  approximately  40%  of  system 
faults. 
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Since  the  figures  fi-om  the  cost-benefit  analysis  are  extremely  positive,  they  should  be  used 
to  gamer  management  support.  In  addition,  the  results  of  recent  system  tests  at  FTC-SD 
strongly  support  the  cost-benefit  analysis  findings. 

C  INVOLVmGMK  92  TECHMCIANS  IN  THE  IMPLEMENTATION 

In  Chapter  HI,  research  had  shown  that  users  should  actively  be  involved  in  the 
implementation  of  an  expert  ^stem  throughout  its  life  cycle.  Enlisting  the  aid  of  technicians 
in  the  development  of  MAES  has  been  pmdent  since  technicians  often  provide  a  different 
perspective  of  a  system.  More  can  be  learned  through  their  participation  than  through 
interviews  or  surveys  alone.  Multinovich  and  Vlahovich  point  out  that  users  have  often  seen 
solutions  that  did  not  work  and  may  have  observed  solutions  that  did  (Multinovich  and 
Vlahovich,  1984).  Without  their  involvement,  valuable  lessons  may  be  lost.  The  MAES 
development  team  has  actively  sought  out  the  participation  of  MK  92  technicians.  It  also 
needs  to  do  so  during  the  implementation. 

1.  Past  Participation  of  MK  92  Technicians 

Throughout  the  development  of  MAES,  prototypes  have  been  given  to  technicians 
for  evaluation  and  suggestions.  The  MAES  program  was  provided  to  the  USS  Sides  (FFG- 
14)  for  two  years  and  the  USS  Wadsworth  (FFG-9)  for  six  months.  They  have  provided 
valuable  ideas  and  suggestions  for  the  system. 

Instructors  at  FTC-SD  and  students  have  also  been  included  in  the  development  of 
MAES.  They  have  played  an  extremely  valuable  role.  Their  suggestions  have  resulted  in  the 
addition  of  more  robust  on-line  help  features;  the  inclusion  of  photos  in  the  help  sections; 
“Why”  buttons  that  provide  insight  into  the  process  the  domain  expert  is  using  to 
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troubleshooting  a  fault;  and  the  inclusion  of  supply  information  that  speeds  up  the  ordering 
process.  Knowledge  by  users  that  thdr  suggestions  are  implemented  in  the  design  of  an  MIS 
stimulates  thdr  enthusiasm  for  the  system  (Bradley  and  Hauser,  1995).  It  has  been  previously 
documented  that  making  such  users  feel  they  are  important  to  the  design  and  evaluation 
process  by  elidting  their  feedback  can  contribute  to  a  successful  implementation  (Lucas- 
Ginzberg,  1990). 

2.  Livolving  MK  92  Technicians  in  the  Deployment  and  Evalnation  of  MAES 

The  dose  relationship  between  MAES  developers  and  the  technician.*;  in  the  fleet  and 
the  instructors  at  the  training  schools  is  one  reason  why  MAES  has  so  far  been  well  received. 
It  is  important  to  continue  this  cooperation  during  the  maiirtenance  of  the  system. 
Approaches  to  maintaining  some  form  of  communication  between  the  developers  and  the 
users  will  be  discussed  in  more  detail  later  in  this  chapter.  Alternatives  to  obtain  input  from 
the  users  include  the  following: 

•  Require  the  technicians  that  have  MAES  to  periodically  submit  a  Digital  Systems 
Feedback  Report  (DSFR).  This  easy  to  fill  out  form  provides  a  convenient  way 
for  technicians  to  provide  suggestions  to  MAES  developers.  The  form  can  be 
either  frxed  or  mailed.  A  copy  is  included  in  the  MAES  User’s  manual  (included 
as  Appendix  C). 

•  Solidtuso"  input  with  periodic  surveys.  A  sample  MAES  user  survey  is  included 
as  Appendix  D.  It  is  discussed  in  more  detail  later  in  this  chapter. 

•  Solidt  responses  from  each  ship  via  standard  navy  radio  message.  While  intrusive, 
this  method  would  ensure  compliance  due  to  its  high  visibility. 

•  Conduct  phone  surveys  of  ships  that  are  using  MAES.  The  disadvantage  of  this 
method  is  that  it  would  be  man-power  intensive  and  would  depend  on  the 
availability  of  technicians  and  whether  or  not  ships  are  inport. 
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The  most  useful  and  cost-effective  method  would  be  to  distribute  a  survey  to  MAES 
equipped  ships  periodically.  A  survey  would  also  provide  some  initial  empirical  data  that 
could  be  further  analyzed.  The  aspects  that  may  be  assessed  would  only  be  limited  by  the 
questions  included  in  the  survey.  Continued  user  involvement  is  important  to  the  successful 
evaluation  of  the  system. 

D.  TRAINING  MAES  USERS 

The  focus  in  this  section  is  training  for  the  initial  implementation  of  MAES.  The 
generic  questions  that  were  raised  in  the  previous  chapter  about  training  are  addressed  for 
MAES  below: 

•  Who  should  be  trained?  All  MK92  fire  control  technicians  and  FTSC  technical 
representatives.  This  includes  those  temporarily  assigned  other  duties  (such  as 
mess  duty). 

•  How  much  training  should  be  done?  One  day  of  classroom  training  with  hands-on 
familiarization  would  be  suffident.  From  feedback  received  from  FTC-SD,  if  they 
were  familiar  with  basic  electronic  test  procedures  and  equipment,  junior 
technidans  were  able  to  use  MAES  within  one  hour  (Myers,  1996). 

•  What  training  methodologies  should  be  used?  Both  classroom  lecture  and  hands- 
on  training.  These  two  methods  provide  the  users  of  MAES  with  the  background 
and  experience  to  effectively  use  MAES. 

•  Who  should  present  the  training?  A  combination  of  instructors  is  recommended. 
For  the  deployment  of  the  initial  implementation,  NSWC-PHD  should  coordinate 
the  training.  NFS  faculty,  &mihar  with  the  history  and  software  development 
should  partidpate.  The  benefit  of  having  a  senior  sailor,  perhaps  from  one  of  the 
training  centers  or  FTSCs,  would  provide  a  tremendous  sense  of  credibility  for  the 
system.  In  addition,  FTSCLANT  personnel  should  participate. 

The  answers  to  the  above  questions  provide  the  framework  for  developing  the  training  plan 

that  will  be  used  by  the  MAES  deployment  team. 
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The  actual  training  provided  MK  92  technidans  will  need  to  cover  sufScient,  detailed 
information  and  hands  on  experience  in  the  use  of  MAES  (Bouldin,  1989).  Following  the 
steps  discussed  in  the  previous  chapter  on  formulating  a  training  plan  provides  an  excellent 
outline  for  proceeding.  The  following  six  sections  discuss  each  step  as  applied  to  MAES 
training. 


1.  Assessing  Training  Needs 

To  assess  training  needs  one  must  determine  who  needs  to  be  trained  and  to  what 
level  each  group  should  be  trained.  Earlier  in  this  chapter  the  shipboard  organizational 
structure  on  a  FFG-7  class  ship  was  defined.  Tailored  training  needs  to  be  provided  for  the 
various  organizational  levels.  FTSC  technical  representatives,  as  well  as  training  center 
instructors,  also  have  to  be  trained.  A  recommended  level  of  training  for  the  different  levels 
in  the  organization  would  include: 

•  Upper  management:  The  commanding  officer,  executive  officer,  and  combat 
systems  officer  should  receive  a  presentation  on  the  capabilities  and  benefits  of 
MAES,  as  well  as  a  short  demonstration  of  the  system.  Approximately  one  hour 
should  be  allotted. 

•  Line  management:  The  division  officer  and  chief  should  attend  the  morning  half  of 
the  technician  training.  They  should  receive  information  on  the  backgroimd, 
benefits,  and  the  basics  of  operating  MAES. 

•  Users:  The  MK  92  technidans  and  FTSC  technical  representatives  should  receive 
the  training  specified  for  line  management.  In  addition,  a  hands  on  training  session 
that  covers  all  aspects  of  MAES  operation  should  be  taught.  For  the  training  in 
the  Norfolk  area,  the  FCTC  facilities  should  be  considered,  as  actual  faults  could 
be  induced  as  part  of  the  training. 

The  goal  of  this  training  is  to  ensure  that  each  organizational  level  understands  the  purpose, 
benefits,  and  limitations  of  MAES  and  that  the  users  are  comfortable  in  its  operation.  The 
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users  and  line  management  should  receive  their  training  in  a  classroom  away  from  the  ship. 
Upper  management  will  receive  their  training  aboard  their  ship.  These  different  levels  of 
training  ensure  that  everyone  receives  training  tailored  to  their  specific  needs. 

2.  Development  of  Training  Material 

The  NFS  has  taken  the  lead  on  developing  the  preliminary  materials.  The  main 
components  of  the  MAES  user  training  program  are  a  training  plan  and  the  MAES  User’s 
Manual.  A  preliminary  training  plan  is  included  as  Appendix  B  and  the  updated  user’s  manual 
is  included  as  Appendix  C.  These  documents  will  be  received  by  NSWC-PHD  and  will  be 
evaluated  after  the  initial  implementation  on  COMNAVSURFLANT  ships.  The  training  plan 
includes  information  about: 

•  The  purposes  and  objectives  of  the  training. 

•  The  required  training  materials  that  the  deployment  team  will  need  to  provide  to 
cany  out  their  training. 

•  A  chronological  outline  of  what  topics  will  be  covered. 

•  A  training  evaluation  form  to  pass  out  to  all  trainees  to  determine  if  the  goals  of 
the  training  are  being  accomplished. 

The  user’s  manual  provides  information  to  users  who  missed  the  training  or  need  to  refresh 
their  knowledge  of  how  to  use  MAES.  The  MAES  user’s  manual  should  contain  the 
following  information:  (Lester,  1996) 

•  An  explanation  of  the  purpose  of  MAES  and  what  information  the  user’s  manual 
provides. 

•  Step  by  step  instructions  on  how  to  install  MAES  and  what  the  hardware  and 
software  requirements  are. 

•  How  to  initialize,  operate,  and  exit  from  MAES. 
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Completing  the  above  documentation  is  the  first  step  in  the  training  process.  The  next  steps 
must  be  complete  before  the  deployment  team  can  progress. 

3.  Training  Team  Composition 

The  training  team  composition  should  minimally  consist  of  two  persons.  One  trainer 
should  be  fi’om  NSWC-PHD,  the  project  manager  for  MAES  and  eventual  system 
configuration  manager.  The  second  member  should  be  an  experienced  Navy  technician  for 
the  MK  92  MOD  2  PCS.  Possible  sources  are  a  FCC  fi’om  FCTC  Dam  Neck  or  an  FCC  fi’om 
FTSCLANT.  An  FCC  should  be  able  to  relate  well  with  his  fellow  sailors.  The  knowledge 
and  job  of  the  FCC  will  also  provide  credibility  to  the  training  team.  Because  this  initial 
deployment  is  for  the  evaluation  of  MAES,  a  feculty  member  fi'om  NFS  should  also  be  part 
of  the  team.  Data  will  need  to  be  gathered  on  MAES  and  NPS  will  coordinate  this  effort. 
The  focus  for  the  NPS  representatives  will  first  be  to  provide  information  on  the  problems 
facing  the  MK  92  MOD  2  technicians  and  the  benefits  MAES  offers  based  on  the  NPS  cost- 
benefit  analysis.  The  second  purpose  will  be  to  implement  the  evaluation  process. 

4.  Preliminary  Training  Review 

While  this  research  has  produced  the  initial  training  components,  review  by  others 
knowledgeable  of  the  MK92  is  warranted.  Therefore,  copies  of  the  training  plan  and  user’s 
manual  should  be  forwarded  to  FTC  San  Diego,  FCTC  Dam  Neck,  FTSCLANT,  and  MAES 
developers  at  NSWC-PHD  for  review.  Feedback  should  be  received  and  acted  upon  prior 
to  conducting  the  actual  training. 
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5.  Evaluation  of  Training 

At  the  end  of  the  training  sessions,  an  evaluation  should  be  distributed  to  the  attendees 
for  their  view  of  training.  An  example  evaluation  is  included  at  the  end  of  the  training  plan 
in  Appendix  B.  Completed  evaluations  should  be  reviewed  carefully  by  the  trainer  at  the 
conclusion  of  each  training  session.  Needed  changes  should  be  made.  A  consensus  of  the 
training  team  members  should  decide  on  needed  changes. 

6.  Alternatives  to  On-Site  Training 

Providing  MAES  users  with  on-she  training  is  the  preferred  method  of  training.  It  has 
the  benefit  of  addressing  questions  on  the  spot,  providing  dedicated  attention  to  a  technidan 
having  trouble  comprehending  some  aspects  of  MAES,  and  a  chance  to  provide  a  personal 
brief  to  the  chain  of  command.  Should  availability  of  ships  or  funding  constraints  pose  a 
problem  with  providing  on-ate  training  to  all  ship’s  crew  there  are  less  expensive  options  that 
could  be  considered.  Options  include; 

•  Training  ship  personnel  via  video-teleconferencing.  This  method  would  save  the 
MAES  deployment  team  travel  costs.  However,  it  would  require  considerable 
coordination  and  may  be  subject  to  time  restrictions  that  are  assodated  with  using 
military  video-teleconferencing  equipment. 

•  A  videotape  could  be  produced  by  the  MAES  deployment  team  and  sent  to  each 
ship  along  with  the  MAES  software  and  hardware.  This  option  has  the  advantage 
of  providing  MAES  users  with  a  well  choreographed  and  structured  training 
sessioa  It  would  not  allow  the  deployment  team  to  address  specific  questions  nor 
address  the  chain  of  command  personally. 

•  Sending  each  ship  printed  training  materials  along  with  the  MAES  software  and 
hardware.  This  is  the  least  preferred  method,  but  provides  an  alternative  if  funding 
becomes  a  problem. 
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•  Provide  training  to  personnel  at  one  central  training  facility  on  the  East  Coast. 
While  this  poses  the  least  costly  alternative  for  the  training  team  (in  terms  of  both 
time  and  money),  it  puts  a  heavy  burden  on  the  fleet  sailors.  Not  all  MK  92  MOD 
2  technidans  would  likely  be  able  to  attend  because  of  the  costs  (in  both  time  and 
money).  The  system’s  goal  is  supposed  to  ease  the  load  on  the  sailor.  This  shifts 
the  burden  fi-om  the  implementation  team  to  the  fleet  and  is  considered  the  least 
desirable  alternative. 

Actual  training  alternatives  may  be  a  combination  of  the  above  methods.  But  the  preferred 
method  would  be  the  on-site  training  of  MAES  users  in  their  homeports  by  a  professional 
deployment  team. 

E.  SUPPORTING  MAES  USERS 

This  section  describes  support  that  should  be  available  to  the  MAES  users  after  the 
system  has  been  deployed.  The  MAES  configuration  manager  will  be  a  project  engineer  at 
NSWC-PHD  (Code  4W32).  The  configuration  manager  will  provide  MAES  users  with  a 
single  point  of  contact  that  is  knowledgeable  in  the  technical  aspects  of  the  system,  both 
hardware  and  software. 

1.  Hardware  Support 

If  a  MAES  computer  suffers  a  defect  that  is  covered  under  the  manufacturer’s 
warranty,  then  the  ship  will  get  the  computer  repaired  through  the  applicable  company’s 
warranty  program.  A  computer  that  sustains  damage  that  is  not  covered  by  warranty,  should 
have  to  be  processed  for  replacement.  The  configuration  manager  at  NSWC-PHD  should 
maintain  a  list  of  acceptable  models  and  manufacturers  available  on  GSA  schedule  or  other 
sources  that  meet  MAES’  minimum  requirements. 


71 


2.  MAES  Software  Support 

Software  support  for  the  MAES  program  falls  into  two  categories:  pre-bundled 
commercial-oflf-the-shelf  software  and  MAES  developed  software.  Pre-bundled  software 
includes  all  software  that  comes  pre-installed  on  the  computers  such  as  the  Windows™ 
operating  system.  MAES  users  will  use  the  provided  documentation  and  commercial  support 
provided  by  the  applicable  software  company  for  pre-bundled  software.  Support  for  MAES 
software  will  be  provided  by  the  MAES  configuration  manager  at  NSWC-PHD.  The  user’s 
manual  provides  MAES  users  with  directions  for  contacting  the  MAES  configuration 
manager.  It  also  provides  reporting  procedures  for  software  problems  and  suggestions. 
When  upgrades  are  needed  they  will  be  made  available  by  FTP  on  the  NSWC-PHD  web  page, 
by  transmission  via  the  Streamlined  Automated  Logistics  Transmission  System  (SALTS),  or 
by  distributing  upgrades  on  diskettes  via  mail  to  each  ship.  If  resources  are  available, 
information  on  MAES  upgrades  and  user  contributions  may  be  available  through  a  MAES 
newsletter  mailed  to  each  ship  or  by  publications  listed  on  the  NSWC-PHD  web  server. 

F.  CONDUCTING  A  USER  SURVEY 

A  user  survey  will  be  conducted  to  determine  the  value  of  MAES  during  the 
evaluation  period.  It  will  be  used  to  detemiine  user  satisfaction,  suggestions  that  MAES  users 
have,  and  for  documenting  recommended  improvements  that  need  to  be  made  to  either  the 
MAES  knowledge  or  user  interface.  The  MAES  user  survey  should  be  conducted  by  mail 
since  this  is  the  most  cost  effective  method.  It  also  ensures  that  aU  ships  can  be  contacted 
whether  inport  or  underway.  In  a  mail  surv^,  the  MAES  users  will  have  more  time  to  collect 
facts,  talk  with  each  other,  or  consider  replies  at  length  than  is  possible  with  a  telephone  or 
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personal  interview.  Another  alternative  would  be  to  use  e-mail  for  responses.  A  proposed 
MAES  user  survey  is  included  as  Appendix  D. 

1.  Implementing  the  User  Survey 

A  primary  consideration  used  in  the  design  of  the  MAES  survey  was  that  the 
respondent  should  be  able  to  answer  the  questionnaire  in  a  short  period  of  time,  e.g.  ten 
minutes  (Cooper  and  Emory,  1995).  The  following  procedures  should  be  used  to  ensure  that 
the  survey  is  returned  by  as  many  respondents  as  possible:  (Cooper  and  Emory,  1995) 

•  Follow-ups,  or  reminders,  should  be  conducted  after  the  survey  is  distributed  in 
order  to  increase  the  response  rate. 

•  Advance  notification  of  the  survey  should  be  given  by  the  deployment  team 
notifying  the  MAES  users  that  they  will  be  asked  to  complete  a  survey  that  is 
important  for  system  improvement  and  will  have  a  significant  impact  on  the  fleet 
wide  deployment  of  MAES. 

•  The  user  survey  should  be  designed  to  be  completed  by  the  user  within  ten 
minutes. 

•  Pre-addressed,  stamped  envelopes  should  be  provided  with  the  surveys. 

•  A  personalized  cover  letter  should  be  included  with  the  survey  telling  the  users  that 
their  opinions  are  important  to  the  improvement  process  and  for  justification  for 
fielding  MAES  to  all  FCS  MK  92  MOD  2  ships. 

•  Users  should  be  informed  that  their  anonymity  is  assured. 

The  survey  should  be  conducted  at  least  twice  during  the  implementation  of  the  initial 
evaluation.  This  will  allow  the  analysts  to  gauge  any  changes  in  user  opinion  or  use  during 
the  initial  implementation  period.  The  piimaiy  research  question  for  the  survey  is  “What  is 
your  level  ofsatisfection  with  the  MAES  software?”  A  subsidiaiy  research  question  is  “What 
improvements  can  be  made  on  MAES?” 
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2.  Analyzing  the  Results  of  the  Survey 

The  majority  of  the  questions  on  the  survey  are  answered  by  the  respondent  circling 
a  number.  This  type  of  design  decreases  the  time  required  to  complete  the  survey  and  malfftg 
the  analysis  of  the  data  easier.  The  target  audience  will  be  all  MAES  users  that  have  actually 
used  MAES  to  diagnose  faults  in  the  MK  92  PCS. 

Methods  of  analyzing  the  data  should  include  running  a  frequency  of  variables  in  a 
statistical  analysis  program,  such  as  SPSS.  A  frequenq^  of  variables  analysis  would  break  out 
which  a^ects  of  MAES  users  like  or  dislike  the  most.  A  histogram  plot  should  be  produced 
to  determine  how  respondents  rate  MAES.  This  analysis  should  also  be  useful  in  determining 
areas  where  the  software  is  not  meeting  the  full  expectations  of  the  users.  Another  analysis 
that  could  be  done  is  an  analysis  of  variances  to  determine  the  correlation  between  how  often 
MAES  is  used  with  satisfaction.  The  analysis  of  variance  test  could  also  show  if  there  is  one 
aspect  of  MAES  that  is  poorly  designed  and  deters  the  user  from  using  MAES.  A  comment 
area  is  included  to  allow  MAES  users  to  elaborate  on  any  of  the  questions  they  were  asked. 
The  analysts  can  categorize  comments  into  categories  such  as  ease  of  use,  suggestions,  or 
hardware  likes/dislikes  to  determine  if  any  suggestions  are  made  often  enough  to  warrant 
attention. 

G.  HARDWARE  IMPLEMENTATION  ISSUES 

Part  of  the  evaluation  of  MAES  will  deal  with  an  assessment  of  the  computer 
hardware  requirements  for  MAES.  During  the  evaluation,  MAES  will  be  deployed  on  sk 
COMNAVSURFLANT  ships  using  COTS  laptop  computers.  One  of  the  first  issues  will  be 
to  determine  if  laptop  computers  are  suitable  for  the  effective  employment  of  MAES. 
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Another  element  in  the  evaluation  will  be  the  reliability  of  COTS  computers.  Other  features, 
such  as  screen  visibility,  user  input  ergonomics,  battery  life,  etc.,  will  also  be  evaluated.  This 
section  discusses  the  subsidiary  research  question  “  What  are  the  hardware  implementation 
issues  assodated  with  deploying  MAES?”  This  subject  was  previously  examined  by  McGaha 
(1994).  The  section  draws  extensively  on  McGaha’s  research.  It  provides  updates  to  his 
recommendations  in  areas  that  have  changed. 

1.  PortabOity 

MAES  provides  a  two-way  communication  with  the  technician  to  diagnose  the 
system.  It  has  been  designed  with  features  that  will  enable  the  technician  to  expeditiously 
diagnose  and  repair  a  fault.  These  include  illustrations  on  how  to  carry  out  a  called  for 
procedure,  pictures,  and  part  number  information. 

Since  the  different  components  of  the  MK  92  PCS  are  located  in  six  different 
compartments  of  the  ship,  a  laptop  computer  will  provide  the  portability  needed  to  use  MAES 
effectively  and  is  strongly  recommended.  McGaha  provided  additional  guidance  on  the  use 
of  dedicated  laptop  computers  that  included:  (McGaha,  1994) 

•  The  computa^  should  be  classified  as  a  piece  of  test  equipment.  This  will  ensure 
minimal  use  of  the  computer  for  administrative  purposes  and  give  first  priority  for 
its  use  to  maintenance. 

•  A  laptop  computer  can  be  stored  more  easily  and  in  a  secure  container. 

•  A  dedicated  hardware  platform  will  minimize  the  potential  for  viruses  and  the 
software  conflicts  that  can  occur  with  other  installed  software. 

•  It  ensures  that  a  technician  has  access  to  MAES  at  any  time,  not  only  to  use  for 
troubleshooting,  but  also  for  training. 
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2.  Commercial-OfT-The-Shelf  (COTS)  Versus  Ruggedized  Computers 

Although  raggedized  computers  are  designed  to  withstand  an  industrial  environment, 

including  many  elements  experienced  by  a  ship  at  sea,  the  features  come  at  a  high  cost.  While 
prices  on  ruggedized  computers  have  decreased  significantly  over  the  past  two  years,  they 
currently  cost  three  to  four  times  as  much  as  their  non-ruggedized  counterparts,  with  prices 
fi-om  $5,000-$10,000.  From  a  personal  perspective,  based  upon  five  years  of  sea  duty,  COTS 
laptops  survive  remarkably  well  onboard  a  navy  ship.  The  Army  has  deployed  an  expert 
system  on  both  COTS  laptops  and  ruggedized  laptops.  They  found  no  major  advantages  for 
the  ruggedized  versions  (Healfinan,  1996).  The  MAES  project  has  always  been  an  austerely 
fimded  project.  And  while  it  is  desirable  to  purchase  both  ruggedized  and  COTS  portables 
for  evaluation,  at  present,  funding  for  ruggedized  computers  is  not  available.  Therefore,  the 
implementation  plan  calls  for  COTS  laptop  computers. 

3.  Cost  and  Support 

Over  the  last  decade  the  cost  for  computer  hardware  has  constantly  been  declining. 
New  generations  of  laptop  computers  are  appearing  approximately  every  six  months.  As  a 
result,  powerful  laptops  are  fi-equently  discounted  40-50%  when  a  new  generation  appears. 
The  aflfordable  laptop  computers  available  today  for  $1500  will  provide  ample  power  to  run 
MAES  efficiently.  The  least  capable  laptop  CPU  that  is  available  at  the  time  this  thesis  was 
written  is  a  486-DX4  75Mhz  chip.  Even  these  types  of  processors  are  becoming  increasingly 
hard  to  find  in  quantity  and  by  the  time  a  purchase  is  made  for  deploying  MAES,  Pentium-75 
MHz  chips  will  most  likely  be  the  cheapest  CPU  available.  One  of  the  lessons  learned  during 
the  course  of  MAES  development  is  that  a  low  cost  computer  may  not  be  the  cheapest 
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alternative  in  the  long  run.  Service  support  for  generic  brand  notebooks  has  not  been  as  good 
or  as  responsive  as  that  of  major  name  brand  manufacturers.  Reliability  has  also  been  a 
problem  with  generic  brand  laptops.  With  todays  more  liberated  acquisition  policy  and 
numerous  contracts  or  alternatives  that  meet  competitive  requirements,  it  is  possible  to 
purchase  high  quality  computers  with  excellent  warranty  coverage.  Indefinite  Delivery, 
Indefinite  Quantity  (IDIQ)  contracts.  General  Services  Administration  (GSA)  schedules,  or 
mail  order  companies  are  all  available  sources. 

H.  SUMMARY 

In  this  chapter,  the  implementation  issues  discussed  in  Chapters  n  and  m  were  applied 
to  the  initial  implementation  of  MAES.  Obtaining  chain  of  command  support  for  MAES  was 
established  as  crudal  to  the  success  of  MAES  when  deployed.  Involving  MK  92  technicians 
in  the  implementation  and  properly  training  all  MAES  users  were  also  examined.  The 
importance  of  supporting  MAES  users  and  conducting  a  user  survey  were  addressed.  Finally^ 
hardware  in^lementation  issues  were  updated  since  they  were  last  examined  by  John  McGaha 
(1994).  The  next  chapter  provides  a  summary  of  conclusions  and  recommendations. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


Deployment  issues  addressed  in  this  thesis  apply  to  the  implementation  of  the  initial 
production  version  of  MAES  on  the  six  Atlantic  Fleet  test  frigates.  This  chapter  summarizes 
the  findings  of  this  research.  It  also  makes  recommendations  for  fielding  MAES  and  follow 
on  research. 

A.  SUMMARY  OF  CONCLUSIONS 

This  thesis  research  was  focused  on  one  primary  research  question  and  four  subsidiary 
research  questions.  The  following  discussion  pertains  to  these  research  questions. 

1.  What  Are  the  Implementation  Issues  for  the  Initial  Fielding  of  MAES? 

Two  different  efforts  provide  a  keen  inaght  to  implementation  issues  that  confront  the 
MAES  deployment.  According  to  Prerau  (1990),  implementing  an  expert  ^stem  is 
composed  of  three  main  phases;  The  initial  phase,  the  core  development  phase  and  the  final 
development  and  deployment  phase.  Each  phase  has  certain  tasks  that  must  be  completed 
before  proceeding  to  the  next  phase.  Following  this  implementation  strategy  ensures  that  an 
expert  ^stem  development  team  will  consider  and  plan  for  applicable  issues  before  the  actual 
implementation  begins. 

The  Lucas-Ginzberg  (1990)  model  of  implementation  examines  implementation  of  a 
managanent  information  system  (MIS)  from  the  perspectives  of  managemftnt  and  the  users. 
Re\dew  of  this  model  provides  insight  to  the  implementation  factors  that  can  influence  either 
management  or  user  acceptance  of  a  new  MIS.  Knowing  which  implementation  factors  will 
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affect  management  and  users  the  most  allows  a  deployment  team  to  better  prepare  for  the 
implementation. 

Drawing  from  the  previous  research  efforts,  the  implementation  factors  that  follow 
will  prove  to  be  critical  for  a  successful  implementation  of  MAES. 

•  Obtaining  management  support.  The  MAES  deployment  team  must  include  all 
levels  of  management  in  their  training.  It  will  be  very  important  to  gain  then- 
support  and  keep  it. 

•  Involving  users  in  the  development.  The  MAES  project  team  should  continue  to 
involve  users  and  listen  to  their  suggestions  and  concerns. 

•  Properly  training  users.  Professional  training  will  be  essential  for  users  to  use 
MAES. 

•  Establishing  a  support  infrastructure  and  effective  lines  of  communication  between 
the  users  and  the  maintenance  team.  Neglecting  this  area  will  turn  many  players 
against  MAES. 

•  Choosing  the  right  hardware  platform.  A  key  reason  for  deploying  MAES  on 
notebook  computers  is  to  support  the  users  need.  Deploying  the  system  on  a 
desktop  computer  could  potentially  turn  users  away  from  using  MAES  if  they 
ejq)erience  delays  getting  access  or  if  needed  features  are  not  av^able  where  the 
work  is  being  done. 

•  Conducting  a  post-implementation  evaluation  to  determine  user  satisfaction  and 
where  improvements  need  to  be  made.  This  should  be  an  ongoing  process  even 
after  Fleet  fielding.  Continual  product  improvement  should  be  ongoing. 

2.  How  Can  Implementation  Issues  Concepts  be  Applied  to  the  Initial 
Deployment  of  the  MAES  Initial  Version? 

Applicable  issues  were  discussed  in  great  detail  in  Chapter  V.  The  deployment  issues 
that  the  MAES  deployment  team  faces  include: 

•  Obtaining  support  from  individual  chains  of  command.  Communications  wU  be 
essential  for  this  issue. 


80 


•  Involving  MK  92  technicians  in  the  implementation  and  further  development  of 
MAES  should  be  ongoing. 

•  Properly  training  all  MAES  users  and  their  respective  chains  of  command  is 
essential.  Deployment  teams  should  have  at  least  one  senior  enlisted  sailor. 

•  Assessing  other  alternatives  to  on-site  training.  The  inconsistency  of  inport 
availability  of  FFG-7s  may  require  one  of  the  alternative  methods  of  training. 

•  Providing  support  for  MAES  hardware  and  software  will  be  essential  during  the 
initial  evaluation  period.  Slow  or  inconsiderate  support  could  turn  the  community 
off  on  MAES. 

•  Distributing  and  ensuring  that  users  respond  to  a  post-implementation  survey  will 
be  essential  for  justifying  deployment  fleet  wide.  Their  responses  will  be  the  most 
heavily  weighted  responses. 

•  Purchasing  hardware  that  balances  value  and  cost  and  that  proves  to  be  reliable  is 
important.  Major  hardware  feilures  that  are  not  dealt  with  promptly,  could  result 
in  an  unfiivorable  rating  for  MAES,  even  though  the  software  performs  admirably. 

3.  How  Can  End-User  Training  Concepts  be  Applied  to  the  Formulation  of  a 
MAES  training  plan  for  shipboard  technicians? 

Chapter  HI  examined  research  that  demonstrates  the  importance  of  properly  training 
users  v^en  implemaiting  expert  systems.  Users  must  not  only  be  trained  on  how  to  use  the 
system,  but  also  how  it  works,  why  it  is  necessary,  and  what  benefits  it  will  provide.  Properly 
assessing  user’s  basic  training  needs  is  the  first  step  in  developing  a  training  plan.  Other 
preliminary  steps  that  need  to  be  completed  before  actual  training  commences  include: 

•  Training  the  personnel  who  will  be  training  the  users. 

•  Developing  training  material  and  methods. 

•  Obtaining  a  preliminary  evaluation  of  the  training  plan  from  involved  parties. 
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The  actual  training  that  will  be  conducted  should  follow  the  training  plan  consolidated  by  the 
deployment  team.  Factors  that  need  to  be  considered  in  the  development  of  the  training  plan 
are: 

•  The  purpose  and  objectives  of  the  training  as  they  pertain  specifically  to  MAES. 
MAES  encounters  the  additional  risks  of  introducing  a  new  technology  into  the 
work  environment. 

•  The  methods  of  training  that  will  be  used  for  each  level  in  the  chain  of  command. 

•  The  level  of  training  appropriate  for  each  level  in  the  chain  of  command.  Training 
should  be  tailored  for  each  group. 

•  The  development  of  an  evaluation  form  that  will  enable  trainers  to  determine  areas 
in  which  to  improve. 

An  effective  training  plan  will  enable  the  deployment  team  to  provide  sufficient,  detailed 
information  and  hands  on  training  to  use  MAES  effectively  and  professionally. 

4.  What  are  the  Hardware  Implemeiitatioii  Issues  Associated  with  Deploying 
MAES? 

The  hardware  issues  associated  with  deploying  MAES  were  addressed  at  the  end  of 
Chapter  V.  First  a  determination  needs  to  be  made  on  whether  a  notebook  computer  needs 
to  be  included  as  part  of  MAES.  For  the  initial  evaluation,  it  is  recommended  that  MAES  be 
pre-installed  on  a  laptop  and  given  to  each  test  ship  and  to  the  FTSCLANT  representatives. 
A  laptop  provides  the  technician  with  a  degree  of  portability  that  is  needed  to  effectively 
utili2e  all  the  features  ofMAES.  A  dedicated  laptop  also  ensures  that  MAES  will  be  available 
whenever  the  technician  needs  it  to  troubleshoot  or  train.  A  dedicated  laptop  also  minimizes 
the  conflicts  that  can  arise  with  other  software  that  may  be  installed  on  the  computer  and 
minimizes  the  introduction  of  viruses  if  users  do  not  load  other  software  programs. 


82 


A  mggedized  laptop  is  not  presently  affordable  for  evaluation.  When  purchasing  the 
laptops,  the  least  e3q)ensive  new  laptops  from  reputable  brand  name  manufacturers  should  be 
purchased.  Affordability  ofa  minimally  equipped  system  is  not  a  problem.  A  laptop  that  has 
the  minimum  MAES  hardware  specifications  is  no  longer  even  available  in  the  marketplace 
unless  it  has  been  pre-owned.  Therefore,  an  affordable  model  from  a  brand  name  that  comes 
with  an  adequate  warranty  should  be  sufficient.  The  method  of  purchase  that  provides  the 
best  price  should  be  used. 

5.  What  are  the  Lessons  Learned  in  the  Implementation  of  Other  Expert 
Systems  for  Weapons  Systems  Within  DOD? 

Two  e:q)ert  systems  that  were  designed  to  diagnose  weapons  systems  have  recently 
been  dq)loyed  in  the  DOD.  The  activities  that  deployed  these  expert  systems  were  contacted 
and  the  recommendations  and  lessons  learned  were  discussed  in  Chapter  IV.  The  key  points 
include: 

•  Training:  a  training  team  that  visits  each  site  was  found  to  be  valuable.  Training 
must  be  coordinated  with  all  involved  commands. 

•  Maintenance:  useful  suggestions  are  made  by  technicians  after  the  systems  were 
dq)loyed.  E3q)ect  to  add  more  on-line  help  and  features. 

•  Post-implementation  evaluation:  technicians  may  not  answer  mailed  surveys. 
Precautions  to  prevent  unretumed  surveys  must  be  taken. 

•  Hardware:  technicians  must  have  access  to  the  system  at  all  times.  COTS  laptops 
are  durable  enough  to  survive  a  maintenance  environment  but  should  not  have  a 
trackball  pointing  device  due  to  its  vulnerability  to  dirt  and  grease. 
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B.  RECOMMENDATIONS 


The  following  subsections  provide  recommendations  for  deploying  the  initial  version 
of  MAES  to  the  designated  test  ships  and  recommendation  for  future  research  in  the 
implementation  of  MAES. 

1.  Obtaining  the  Support  of  Upper  Management 

Upper  management  support  has  been  proven  to  be  one  of  the  strongest  factors 
affecting  an  expert  system’s  successful  implementation.  Therefore,  special  attention  needs 
to  be  paid  to  the  effective  presentation  of  the  capabilities  and  benefits  that  MAES  can  provide 
to  each  ship.  The  presentation  should  highlight  key  findings  in  Powell’s  (1993)  cost  benefit 
analysis  to  underscore  the  benefits  available.  A  concise  brief  with  a  short  demonstration 
should  be  made  to  each  ship.  A  MAES  advocate  fi-om  either  FTC-SD,  FCTC  Dam  Neck,  or 
FTSC  should  accompany  the  deployment  team  to  provide  technical  backing  to  the 
presentation.  Management  will  look  for  sailor  credibility  and  championing  in  order  to  fully 
support  MAES  themselves. 

2.  Involving  the  MK  92  Technicians  in  the  Implementation  of  MAES 

The  opportunity  for  getting  MK  92  technicians  involved  occurs  during  deployment 
training.  During  the  training  sessions  the  deployment  team  can  highlight  the  contributions 
other  technicians  have  made  and  the  features  in  MAES  that  resulted.  If  resources  are 
available,  it  is  also  recommended  that  a  MAES  newsletter  be  distributed  periodically  to  inform 
all  MK  92  technicians  about  their  contributions. 
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3.  Training  MAES  Users 

The  training  of  MAES  users  should  be  conducted  by  a  deployment  team  that  travels 
to  each  training  site.  This  method  of  training  ensures  that  all  MAES  users  on  the  test  ships 
will  receive  sufficient  quality  training.  Hands  on  exercises  should  be  included  in  the  training. 
The  training  plan  provided  as  Appendix  B  may  be  used  as  a  basis  for  the  actual  training.  The 
training  evaluation  forms  should  be  passed  out  at  the  conclusion  of  each  training  session  and 
carefully  read  to  determine  if  changes  to  the  training  plan  need  to  be  made. 

4.  Providing  Support  to  MAES  Users 

A  MAES  newsletter  and  telephone  support  line  should  be  available  to  aid  users.  The 
personnel  that  will  be  answering  the  phone  will  need  to  be  knowledgeable  in  both  the  MK  92 
FCS  and  the  MAES  software.  Additionally,  a  MAES  world  wide  web  page  could  be 
constructed  in  order  to  provide  answers  to  frequently  asked  questions  and  provide 
downloadable  software  updates.  Open  communications  and  prompt,  carefiil  follow-up  to 
each  inquiry  will  be  essential  for  maintaining  user  confidence  in  MAES  support. 

5.  Conducting  and  Analyzing  the  Results  from  the  User  Survey  to  Determine 
Fleet  Suitability 

There  will  be  no  way  to  determine  if  MAES  users  are  satisfied  if  a  survey  is  not 
conducted.  The  methods  discussed  in  Chapter  in  to  increase  user  response  to  the  survey 
should  be  used.  Once  the  surv^s  are  received,  the  data  should  be  analyzed  to  determine  user 
satisfaction  and  whether  improvements  are  necessary.  Continual  testing  of  the  system  at  the 
training  centers  can  also  provide  more  empirical  evidence  to  compare  with  projected  benefits. 


85 


6.  Utilizing  COTS  Hardware 

Because  it  is  possible  to  get  computers  with  three  year  warranties,  and  the  relatively 
low  cost,  it  is  recommended  that  the  initial  issue  MAES  laptop  be  classified  as  a  consumable 
item.  If  repair  is  beyond  an  economic  price,  the  ship  wiU  be  responsible  for  replacing  it.  All 
of  this  discussion  is  based  on  a  premise  that  the  initial  evaluation  will  find  it  beneficial  to 
deploy  MAES  on  a  notebook  computer. 

Since  the  least  capable  laptop  that  is  commercially  available  as  of  the  writing  of  this 
thesis  exceeds  the  MAES  minimum  hardware  requirements,  it  is  recommended  that  the 
MAES  laptops  be  bought  with  cost  as  the  primary  consideration.  The  manufacturer  should 
be  a  brand  name  that  is  reputable  and  well  rated  in  current  computer  industry  magaytnftg 
while  also  providing  a  competitive  warranty.  The  model  that  is  purchased  should  also  have 
at  least  a  10.4"  color  dual  scan  technology  screen  to  ensure  that  help  photos  are  displayed 
with  sufficient  resolution. 

NPS  has  evaluated  the  input  pointing  devices  on  notebook  computers.  The  device 
should  be  integral  to  the  computer.  An  attachable  mouse/pointing  device  is  not  acceptable. 
Attaching  and  detaching  will  pose  a  reliability  issue  over  time.  It  also  may  be  lost  or  could 
more  easily  be  broken  than  an  integrated  device.  Trackball  pointing  devices  tend  to  get  dirty 
and  are  not  appropriate.  The  touchpad  pointing  device  has  become  popular  on  several 
computers.  Its  solid  state  design  has  proven  reliable.  However,  users  who  used  it  did  not  like 
the  “feel”  or  the  saisitivity.  Its  effectiveness  also  may  pose  a  problem  if  dirt  and  grease  build 
up  on  it.  The  preferred  device  was  the  trackpoint  stick  first  used  in  the  IBM  Thinkpad 
computers. 
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The  minimum  requirements  for  a  MAES  laptop  are: 

•  486DX2  66MhzCPU 

•  SMB  RAM 

•  10.4"  Dual  Scan  Color  Screen 

•  340  MB  Hard  Drive 

•  Integrated  Trackpoint  pointing  device 

•  3.5"  1.44  MB  floppy  drive 
C.  FOLLOW  ON  RESEARCH 

Currently,  MAES  is  being  evaluated  by  the  MK  92  Mod  2  PCS  instructor  staff  at 
FTC-SD.  MAES  is  also  planned  to  be  evaluated  on  sk  Atlantic  Fleet  ships  in  November.  A 
survey  should  be  distributed  at  least  twice  during  the  evaluation  period.  Follow  on  research 
could  include  an  analysis  of  the  data  received  fi-om  the  survey  to  determine  whether  MAES 
users  are  satisfied  and  what  improvements  need  to  be  made. 
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APPENDIX  A.  IMPLEMENTATION  PLAN  FOR  DEPLOYING  THE  INITIAL 

VERSION  OF  MAES 


Task 

Provide  inputs  to  fleet  scheduling  conference  to  arrange  test  ships  for 
deployment. 

Identify  personnel  for  the  deployment  team. 

Establish  a  Deployment  Team  meeting  schedule  to  coordinate 
preparations  and  make  assignments. 

Establish  a  checklist  to  identify  the  problems,  date  and  specific  person  to 
ensure  all  problems  that  come  up  are  identified  and  solved. 

Ensure  all  documentation  applicable  to  the  deployment  are  available  and 
complete. 

-  Training  Plan/Evaluation 

-  Implementation  Plan 

-  User’s  Manual 

Purchase  required  number  of  laptop  computers. 

Send  training  plan  to  evaluating  commands  for  review. 

Ensure  that  all  members  of  the  deployment  team  who  will  be  conducting 
training  are  capable  of  teaching  the  required  topics. 

Obtain  training  site  fi’om  local  command  in  charge. 

Draft  radio  message  informing  participating  commands  of  locations  and 
times  for  training  and  briefs. 

Arrange  lodging  and  transportation  for  the  deployment  team. 

Furnish  required  security  clearances  for  deployment  team  access  to 
applicable  commands. 

Bmld  PowerPoint  slide  show  for  individual  command  presentations  (45 
min.  duration). 

Deployment  team  reviews  training  plan. 

Configure  laptops  to  optimally  run  MAES  (install  MAES,  video  settings, 
etc.) 
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Task 

Due  Date 

Ensure  that  all  required  training  materials  are  assembled  (See  training 
plan). 

Deploy  MAES. 

Document  lessons  learned. 

Distribute  user  surveys. 
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APPENDIX  B.  MAES  TRAINING  PLAN 


Training  will  consist  of  two  days  at  each  site.  The  first  day  will  consist  of  classroom  training 
at  a  local  training  facility  (FTC  in  Norfolk  and  base  classroom  in  Mayport).  The  second  day 
will  consist  of  a  brief  visit  to  each  ship  to  brief  the  chain  of  command  and  conduct  follow-up 
training  on  MAES  users. 

First  Day 

Training  session  to  commence  at  0800  on  the  first  day. 

Required  materials:  Overhead  projector,  training  slides,  student  binders  that  contain  copies 
of  training  slides  and  training  evaluation  forms,  canned  scenarios  for  MAES  training,  MAES 
HAV  and  S/W  for  users  (one  per  ship),  associated  MAES  documentation  for  users,  and 
MAES  laptop  for  instructor. 

Required  attaidees:  Ordnance  OflBcer  (first  hour).  Division  Chief,  Recommended  that  ships 
send  all  MK92  technicians. 

Training  Syllabus: 

1.  Background  of  MAES: 

A.  What  is  an  expert  system 

B.  The  basics  of  an  expert  ^stem 

1)  Components:  Interface,  Database,  Inference  Engine 

2)  Expert:  Knowledge  acquisition,  who  he  is 

3)  The  Technician:  Most  important  part  of  the  system 

4)  MAES  Capabilities 

a.  What  it  can  and  cant  do 

b.  Discuss  results  of  FTC-SD  evaluation 

C.  How  MAES  is  different  fi’om  tech  manuals 

1)  70%  new  knowledge 

2)  Knowledge  behind  MAES  is  fi-om  a  UNISYS  engineer  with  30  years 
experience 

D.  Why  the  system  is  important 

1)  It  WU  Save  You  Time! 

2)  It  will  save  money! 
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2.  Intro  to  MAES 


A.  Hardware 

1)  Proper  care 

2)  Basic  operation 

3)  Custody  forms 

4)  Repair  procedures 

-  Who  to  contact  for  problems  with  the  S/W  or  HAV 


B.  Software 

1)  Installation 

2)  User's  Manual  contents 

LUNCH 
3.  MAES  Operation 

A.  Navigating  the  program 

1)  Opening  the  program 

2)  Going  from  screen  to  screen 

B.  Canned  Scenarios 

1)  Go  through  scenarios  step-by-step 
-  One  easy  and  one  hard 

C.  Fill  out  training  evaluation  forms 
Second  Dav 

Will  consist  of  a  ship  visit  to  each  ship  that  MAES  will  be  evaluated  on.  A  space  with  an 
overhead  to  brief  the  chain  of  command  on  each  ship  will  need  to  be  pre-arranged.  The  dates 
and  times  of  these  briefs  will  have  to  be  forwarded  to  the  ships  as  early  as  possible  so  they  can 
schedule  them. 

1.  Short  (1/2  hr)  brief  to  chain  of  command  (CO,XO,CSO) 

OUR  CHANCE  TO  GET  MANAGERIAL  SUPPORT  FOR  MAES 

A.  What  MAES  does 

B.  How  it  will  save  the  sailor  time,  make  his  job  easier  and  save  the  ship  money 

C.  Display  Cost/Benefit  analysis  findings 
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D.  Convince  the  command  of  the  importance  of  using  the  system 

-  As  part  of  the  evaluation  they  are  extremely  important  as  they  will  be 
putting  the  system  through  its  paces 

2.  Brief  follow  on  visit  with  the  technicians  to  see  if  they  have  any  last  minute  questions. 
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TRAINING  EVALUATION  FORM 


This  evaluation  form  is  provided  for  you  to  fill  out  in  order  that  we  can  provide  you  with 
high  quality  trdning.  We  ask  for  your  cooperation  in  filling  out  this  short  questionnaire. 
It  should  take  no  more  than  five  minutes  of  your  time.  Providing  us  with  your  name  is 
optional  only.  Please  turn  the  completed  questionnaire  in  to  a  member  of  the  MAES 
deployment  team  when  you  are  done.  Thank  you  very  much  for  your  assistance. 

Instructions:  Please  circle  the  appropriate  response  and  provide  comments,  if  any,  in  the 
space  provided  at  the  end  of  the  questionnaire. 

1 .  Was  your  training  room  comfortable? 

Yes  No 

2.  Did  the  instructor  tell  you  what  the  training  objectives  were? 

Yes  No 

3.  Do  you  feel  you  have  an  imderstanding  of  what  MAES  can  and  cannot  do? 

Yes  No 

4.  Do  you  understand  how  to  install  MAES  fi-om  the  floppy  disks? 

Yes  No 

5.  Do  you  know  who  to  contact  in  the  event  you  have  a  problem  or  question  about 
MAES? 

Yes  No 

6.  Do  you  understand  how  to  troubleshoot  with  MAES? 

Yes  No 

Comments: 
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APPENDIX  C.  MAES  USER’S  MANUAL 


MK  92  MOD  2  FCS  MAINTENANCE  ADVISOR 
EXPERT  SYSTEM  (MAES) 


USER’S  MANUAL 
VERSION  1.0 
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Chapter  1 


Welcome  to  MAES 


Introducing  MAES 

The  MK  92  Mod  2  Maintenance  Advisor  Expert  System  (MAES)  was  developed  to  assist  fire 
control  technicians  in  the  fleet  to  correctly  diagnose  casualties  in  the  MK  92  Mod  2  Fire 
Control  System.  MAES  will  increase  your  operational  readiness  by  giving  you  “expert” 
knowledge  when  and  where  you  need  it  most.  MAES  will  assist  in  locating  faults  discovered 
in  radar  system  Performance  or  Calibration  during  the  Daily  Systems  Operability  Test 
(DSOT).  The  capability  to  diagnose  RF  Power  Checks  will  be  included  in  a  later  version. 

MAES  was  designed  with  the  following  goals  in  mind: 

•  Improving  your  workload  by  decreasing  the  amoimt  of  time  you  spend  diagnnging 
faults  in  the  system. 

•  When  required,  the  ability  to  communicate  with  shore  based  maintenance 
organizations  (such  as  FTSC)  more  eflFectively  concerning  problems  encountered. 

•  Provide  you  with  a  training  tool  for  brushing  up  your  troubleshooting  skills. 

•  Inaeasing  ship’s  state  of  operational  readiness  by  decreasing  the  amount  of  time 
required  to  repair  the  MK  92  FCS. 

•  Eliminating  circuit  cards  that  are  turned  in  that  have  No  Fault  Evident  (NFE). 

•  Saving  ships  money  through  reduced  ordering  of  repair  parts. 

The  knowledge  that  is  behind  the  power  of  MAES  comes  fi’om  senior  engineers  with  many 
years  experience  in  diagnosing  the  MK  92  Mod  2  FCS. 


Note:  This  manual  assumes  you  are  familiar  with  your  computer’s  basic  operations.  If  you 
are  unfamiliar  with  terms  like  “dialog”  or  “double-click”,  or  basic  file  handling  and 
printing  procedures,  please  read  the  owner’s  manual  for  your  computer  anH  your 
Microsoft  Windows/Windows  95  User’s  Guide  before  using  MAES. _ 
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Chapter  2 


Installation 


Reviewing  Your  Package  Contents 

Before  installing  MAES,  please  take  a  moment  to  verify  that  your  package  contains  the 
appropriate  items  and  that  your  computer  meets  the  system  requirements  for  MAES.  Your 
MAES  package  should  contain  the  following  items: 

Two  MAES  Installation  disks,  3.5"  1.44  MB 

PCS  MK92  Maintenance  Advisor  Expert  System  User’s  Manual 

If  this  is  your  initial  receipt  of  MAES  you  should  also  be  in  receipt  of  a  laptop  computer. 
Please  refer  to  the  documentation  that  came  with  the  computer  for  it’s  care  and  maintenance. 

If  any  items  are  missing,  please  contact  MAES  HelpLine  immediately  at  (805)  982-0141  or 
DSN  551-0141. 

Checking  System  Requirements 

MAES  disks  are  not  bootable;  they  do  not  contain  MS-Windows  software.  To  operate 
MAES,  you  need  the  following  hardware  and  software. 

MAES  is  operational  tmder  Windows  3.xx  or  Windows  95.  The  system  requirements  for 
MAES  follow: 

•  IBM  PC  or  100%  compatible 

•  Intel  80386  DX  or  higher  (486  or  higher  recommended) 

•  hftcrosoft  Windows  3 .  lx  or  Windows  95 

•  A  hard  disk  drive  with  at  least  30  MB  of  free  storage  space 

•  Minimum  4  MB  RAM  (8  MB  RAM  recommended) 

•  A  3.5  inch  1.44  MB  floppy  drive 

_ •  A  standard  VGA  monitor  that  supports  640  x  480  resolution  and  256  colors 

Note:  Less  capable  monitors  may  be  used,  but  resolution  quality  may  be  compromised.  If 
you  have  a  monitor  that  is  17"  or  larger,  or  a  graphics  card  with  more  than  2  MB  of 
_ video  RAM,  you  may  achieve  better  results  with  a  resolution  of  800x600. _ 

•  Mouse  or  built-in  pointing  device 
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Installing  MAES 


Installing  MAES 


Installation  takes  about  fifteen  minutes. 

Before  you  install  MAES,  please: 

•  Close  other  applications 

Closing  applications  (such  as  a  word  processor  or  spreadsheet  program)  will  fi’ee 
memory  and  prevent  possible  conflicts  during  installation. 

•  Turn  off  virus  protection  programs 

Some  virus  protection  programs  can  interfere  with  the  installation  process.  Please 
refer  to  the  user’s  manual  for  your  virus  protection  software  for  information  on 
how  to  turn  off  your  virus  protection  program.  We  also  recommend  turning  off 
any  installed  memory-resident  programs  (such  as  a  screen  saver  or  a  pop-up 
program  that  appears  when  you  type  certain  key  combinations). 

•  Make  a  backup  of  your  MAES  program  disks 

See  Making  a  Backup  of  your  MAES  Disks  at  the  end  of  this  chapter  for  more 
information. 

Once  you  have  closed  all  other  ^^^dows  applications,  disabled  virus  protection,  and  made 
a  backup  of  your  MAES  program  disks,  you  are  ready  to  install  MAES.  Follow  these  simple 
steps  to  install  MAES. 

Note:  The  following  instructions  assume  your  hard  disk  is  drive  C  and  your  floppy  disk  is 
drive  A.  If  named  differently,  please  use  your  drive  designators. 

1.  Insert  the  installation  disk  1  of  2  in  a  3.5"  drive. 

For  Windows  3.1x  users: 

2.  Choose  Run  firom  the  Program  Manager  File  menu. 

3.  Type  the  command  AnsrSTALIT.EXE  and  click  OK.  Go  to  step  8. 

For  Windows  95  users: 

2.  From  your  desktop,  double-click  the  My  Computer  icon. 

3.  Double-click  the  Control  Panel  icon. 

4.  Double-click  the  Add/Remove  Programs  icon. 

5.  In  the  Add/Remove  Programs  Properties  window,  choose  the  Install/Uninstall  tab.  Click 

Install. 

6.  Click  Next. 

7.  The  Command  line  in  the  installation  window  should  read  A:\TNSTAT  .tt  f.yf  click 
Finish.  Go  to  step  8. 
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Installing  MAES 


Installing  MAES  (cont) 

8.  The  install  program  will  display  a  panel  telling  you  that  it  is  copying  the  install  program 
to  a  temporary  area  on  your  hard  drive. 

9.  The  install  program  will  display  a  panel  telling  you  about  itself. 

10.  The  install  program  will  begin  displaying  windows  prompting  you  for  responses. 

11.  When  you  are  prompted  to  Select  Option,  select  Install  MAES  1.0  and  click  on  OK. 

12.  When  you  are  prompted  to  Select  hnstallation  Drive  select  a  drive  from  those  listed  that 
has  at  least  30  MB  free  space. 

13.  You  will  be  prompted  to  specify  an  installation  path.  The  recommended  installation 
directory  is  named  MAES  10.  The  recommended  installation  drive  is  where  Windows 
is  installed.  For  example,  if  you  have  Windows  installed  on  drive  C,  C:\MAES10  is  the 
suggested  installation  directory.  Enter  a  different  path  if  you  desire  and  then  press  OK. 

Warning:  Do  not  specify  the  \Windows  or  \Windows\System  directories  as  the  MAES 

directory.  However,  C:\Windows\MAES10  is  acceptable. 

14.  At  this  point  the  install  program  will  begin  copying  files  to  your  hard  drive.  A  progress 
meter  will  show  you  the  status  of  the  installation  throughout  this  process  and  will  prompt 

you  for  a  disk  change. 

15.  When  prompted  to  Install  Icon,  select  the  second  option  Install  Icon  in  a  New  Group 
and  press  OK  unless  you  prefer  to  place  the  icon  in  an  existing  group. 

16.  The  Install  MAES  1.0  menu  will  reappear.  Select  Exit  Installation  and  click  OK. 

17.  Windows  3. lx  users  should  find  the  new  icon  MAES  1.0  in  the  MAES  1.0  program 
group  and  can  run  it  by  double-clicking  on  it.  Windows  95  users  can  run  MAES  by 
clicking  on  Start  -  Programs  -  MAES  1.0  -  MAES  1.0. 

18.  Remember,  at  this  time  only  the  Calibration  and  Performance  options  are  available  for 
use. 


MAES  is  now  ready  to  use! 


Installing  MAES 


Making  a  Backup  of  Your  MAES  Disks 

To  make  a  backup  of  your  MAES  program  disks,  simply  follow  these  steps: 


Note:  The  source  and  destination  disks  must  be  of  the  same  size  (3.5")  and  capacity 
(1.44  MB) _ 

Windows  3.  lx  users: 

1.  Insert  the  source  disk  (MAES  program  disk). 

2.  Open  the  file  manager  and  click  on  Disk. 

3.  From  the  drop  down  menu,  select  Copy  Disk. 

4.  When  prompted,  insert  a  blank  disk  to  copy  onto. 

5.  Repeat  steps  1-4  for  subsequent  MAES  disks. 

Windows  95  users: 

1.  Open  My  Computer. 

2.  Click  the  icon  for  the  disk  you  want  to  copy  (A:  or  B:) 

3.  Select  File  [  Copy  Disk  fi-om  the  My  Computer  menu  bar.  The  copy  Disk  dialog  will  be 
displayed. 

4.  Select  the  Copy  fi-om  (source)  drive  and  the  Copy  to  (destination)  drive.  These  drives  can 
be  the  same.  For  example,  both  can  identify  the  A:  drive. 

5.  Choose  the  Start  button.  Follow  the  on-screen  instructions. 
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MAES  Basics _ 

MAES  is  extremely  flexible  and  powerful.  The  interface  was  designed  to  be  as  simple  as 
possible  so  you  can  access  the  knowledge  quickly  and  effectively.  This  chapter  introduces 
the  basic  prindples  and  procedures  you  need  to  understand  to  work  effectively  and  efficiently. 


Screen  Layout  _ 

The  standard  MAES  screen  is  divided  into  three  primary  sections.  These  sections,  as  shown 
in  Figure  3-1,  are  the  title  bar,  procedure  area,  and  action  area. 


Figure  3-1.  MAES  Standard  Screen  Layout 


Title  Bar 

The  MAES  title  bar  is  always  positioned  horizontally  across  the  top  of  the  screen.  The  title 
bar  displays  information  about  where  you  are  in  the  MAES  program.  For  example,  the  title 
bar  in  Figure  3-1  says  “All  CAS  Track  Modes  FF  and  FA”  and  below  it  “Power  High 
Occurs”.  This  indicates  that  you  are  troubleshooting  a  problem  with  the  CAS  Track  Modes 
FF  and  FA  and  are  currently  determining  whether  a  Power  High  condition  exists.  The  title 
bar  will  change  as  you  move  from  one  area  of  the  program  to  another. 
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Procedure  Area 

The  Procedure  Area  is  the  central  portion  of  the  MAES  display.  In  Figure  3-1  it  is  the 
horizontal  bar  labeled  “Procedure”.  MAES  uses  this  area  to  display  instructions,  procedures, 
and  questions. 

Action  Area 

This  area  is  located  in  the  bottom  area  of  the  screen.  In  Figure  3-1  it  is  the  horizontal  bar 
labeled  “Action”.  This  is  the  area  that  allows  you  to  interact  with  MAES  by  clicking  on 
buttons.  In  Figure  3-1,  the  buttons  that  are  available  are  “Yes”  and  “No”. 

Screen  Identification  Number  (SIN) 

At  the  lower,  right-hand  comer  of  every  MAES  screen  there  will  be  a  small  number.  For 
example,  in  Figure  3-1  it  is  “13".  Refer  to  this  number  and  the  Title  Bar  if  you  have  questions 
about  MAES  or  are  submitting  a  Software  Feedback  Form. 


Input  Methods _ 

The  MAES  display  screens  provide  a  variety  of  methods  to  interact  with  the  program.  The 
most  common  methods  are  buttons,  list  boxes,  and  check  boxes.  These  objects  are  used  to 
teU  MAES  what  you  want  it  to  do  in  the  way  of  displaying  results,  instmctions,  or  help. 

Buttons 

Inputs  to  MAES  are  performed  by  clicking  on  buttons.  For  example,  when  the  button  labeled 
“Prev  Screen”  in  Figure  3-1  is  pressed,  MAES  will  display  the  previous  screen.  Buttons 
appear  on  almost  all  of  the  MAES  screens.  You  can  only  select  one  button  at  a  time. 
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Figure  3-2.  MAES  Screen  Using  List  Box. 


List  Boxes 

List  Boxes  are  another  way  to  interact  with  MAES.  List  Boxes  display  a  list  of  choices  that 
you  may  select  one  option  from.  If  the  list  box  has  more  choices  than  can  be  displayed  in  the 
box,  then  you  can  use  the  scroll  bars  to  view  all  of  the  contents.  Only  one  item  in  a  list  box 
can  be  selected  at  a  time.  Figure  3-2  is  an  example  of  a  MAES  list  box. 


MAES  Basics 


Figure  3-3.  MAES  Screen  Using  Check  Boxes 


Check  Boxes 

A  check  box  allows  you  to  select  or  clear  an  option.  You  can  select  as  many  options  as  are 
available  by  checking  their  corresponding  check  boxes.  A  is  placed  in  a  check  box  when 
it  has  been  selected.  An  example  of  check  boxes  in  MAES  is  shown  in  Figure  3-3.  Notice 
that  six  check  boxes  in  the  left  column  have  been  selected. 
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Figure  3-4.  MAES  Screen  with  Help  Buttons 


Help  Information 

There  are  three  types  of  help  information  available  to  you  in  MAES.  These  types  include 
“How”,  “Why”,  and  “Parts  Info”.  Please  refer  to  Figure  3-4  during  the  discussion  on  the  help 
features. 


How 

Clicking  on  a  “How”  button  gives  you  a  detailed  description  of  how  to  perform  a  required 
procedure.  Once  you  press  a  button  labeled  “How”  in  the  procedure  area,  a  new  screen 
appears  with  the  information  you  requested.  If  there  is  more  information  than  can  be 
displayed  in  one  screen,  it  will  appear  in  a  scrollable  window. 


105 


MAES  Basics 


Photo 

At  the  bottom  of  some  “How”  screens,  this  button  may  be  available.  Clicking  on  the  “Photo” 
button  gives  you  a  photo  or  series  of  photos  that  step  you  through  a  procedure  that  you  may 
need  more  inJformation  on.  These  photos  will  appear  best  if  your  monitor  is  set  for  640x480 
resolution  with  256  colors.  You  may  always  click  on  the  “Return”  button  to  go  back  to  the 
Procedure  screen  where  you  left  off. 


Why 

Clicking  on  a  “Why”  button  provides  you  with  an  explanation  of  why  MAES  is  directing  you 
to  perform  a  certain  task.  This  “Why”  information  gives  you  insight  into  how  the  MK  92 
expert  we  used  to  design  MAES  attacks  a  problem.  Once  you  finish  reading  the  explanation, 
press  “Return”  and  MAES  wUl  take  you  back  to  the  previous  screen. 


Parts  Info 

By  clicking  on  a  “Parts  Info”  button  MAES  will  give  you  parts  information  with  regards  to 
the  part  that  MAES  is  recommending  you  replace.  The  parts  information  will  provide  you 
with  a  part  number,  reference  diagram  number,  if  and  where  the  part  is  used  elsewhere  in  the 
system,  part  designation,  and  the  power  requirement  of  that  part. 

Help  Buttons 

Clicking  on  “Help”  buttons  within  MAES  will  provide  you  with  specific  help  about  the  screen 
you  are  currently  viewing. 
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CA&  Track  Target  Wodes 

Okkl^Ul  PkMvddt  tiOt  Stt.  Tol«ttttAi::i& 


^  O^ireric  Circuit  Caird  Replacement: 


m  1 .  Remove  and  inspect  card/cards  fbr  evidence  of  damage, 

|2.  Replace  damaged  card/cards.  (See  Paris  inforroatton  page.) 


I 


3.  If  inspection  was  o.  k.  reinstall  card/cards  and  recheck  prevfoits  test 
(Ensure  card/cards  are  seated  property.) 

I.  If  test  sBII  fails,  replace  card/cards  one  at  a  dme  beginning  with  the  first 
card  listed  (most  aptto  faifi  and  recheck  test. 

S.  When  available,  use  identical  card/cards  from  other  locations  as  test 
replacement  cards.  (See  Pate  Information  page.) 

5.  ffteStsffli  fails  after  all  cards  have  been  replaced,  check  circuit  associated 
v«th  cards,  (See  Pate  Infbrmafion  page.) 


J 

I 

i 


"""^ 


Figure  3-5.  Circuit  Card  Replacement  Screen. 


Circuit  Card  Replacement 

CAUTION:  It  is  very  important  to  perform  the  generic  part  replacement  methodology 
when  replacing  parts. 

Most  drcuit  cards  in  the  MK  92  Mod  2  FCS  can  be  replaced  using  a  generic  card  replacement 
procedure.  Certain  rules,  which  are  intuitive  to  Fire  Control  Technicians,  must  be  followed 
when  repladng  a  circuit  card.  Procedures  for  generic  card  replacement  procedure  and  rules 
to  follow  after  card  replacement  are  available  in  MAES  as  depicted  in  Figure  3-5. 
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Rules  for  Exiting  MAES  After  Part  Replacement 

The  rules  that  should  be  followed  after  a  part  is  replaced  are  as  follows: 

1 .  If  an  adjustment  is  performed  in  the  path  where  the  part  is  to  be  replaced,  exit  and  perform 
the  adjustment  again. 

a.  If  the  adjustment  can  be  performed  within  specifications,  rerun  DSOT. 

b.  If  the  adjustment  still  can  not  be  performed  within  specifications,  return  to  the 
display  screen  for  part  replacement  and  replace  the  next  part  listed.  Perform  the 
adjustment  again. 

c.  If  all  parts  have  been  replaced  and  adjustment  is  still  out  of  specifications,  return 
to  the  part  replacement  screen.  Obtain  the  figure  reference  for  the  signal  flow 
diagram  associated  with  parts  and  continue  with  troubleshooting. 

2.  If  an  adjustment  is  not  performed  in  the  path  where  the  part  is  replaced,  exit  to  the 
submenu  display  screen  and  check  to  see  if  the  problem  is  corrected. 

a.  If  the  problem  is  corrected,  rerun  DSOT. 

b.  If  the  problem  is  not  corrected,  return  to  the  part  replacement  screen  and  replace 
the  next  part  listed.  Check  and  see  if  the  problem  is  corrected. 

c.  If  all  parts  have  been  replaced  and  the  problem  stUl  exists,  return  to  the  part 
replacement  screen.  Obtain  the  figure  reference  for  the  agnal  flow  diagram  associated 
with  the  part  and  continue  troubleshooting. 


Note:  When  returning  to  the  adjustment  screen  or  submenu  display  screens,  make  certain 
that  the  initial  setup  is  completed  before  performing  an  adjustment  or  when  checking 
_ to  see  if  the  problem  is  corrected. _ 
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The  following  sample  session  is  an  example  of  how  to  use  MAES  for  troubleshooting  a  fault. 
The  example  assumes  a  failure  in  the  Calibration  module  which  has  failure  in  all  CAS  Track 
modes. 

Begin  MAES 

Start  MAES  by  following  the  steps  previously  discussed  at  the  end  of  the  Installing  MAES 
section.  To  begin  MAES,  click  on  the  button  labeled  ‘TBegin  Program”  on  the  opening  screen 
with  the  picture  of  a  FFG-7  class  ship. 

Select  A  Module 


The  next  screen  that  appears  will  be  the  MAES  Main  Menu  and  is  shown  below  in  Figure  4-1 . 
Since  this  example  is  assuming  a  Calibration  CAS  Track  failure,  press  the  Calibration  button 
on  the  left  side  of  the  Action  area. 


Figure  4-1.  MAES  Main  Menu 
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Figure  4-2.  Calibration  Menu 

Once  the  Calibration  button  is  pressed,  the  Calibration  Menu  that  is  depicted  in  Figure  4-2 
will  be  displayed. 


Select  DSOT  Entry  Method 

Two  options  are  available  for  entering  DSOT  data  on  the  Calibration  Menu:  Printout  and 
Manual.  The  Printout  option  allows  you  to  select  problem  areas  on  a  display  screen,  shown 
in  Figure  4-3,  that  mimic  a  DSOT  printout.  Click  on  the  button  labeled  “Printout” 
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Figure  4-3.  Calibration  DSOT  Printout  Menu 


Select  Failed  Areas 

The  next  step  is  to  select  the  failed  areas  by  using  check  boxes  in  the  Printout  Menu.  Check 
the  six  boxes  shown  in  Figure  4-3  that  are  labeled  CAS  TR  TGT  FF,  CAS  TR  CLT  FF,  CAS 
TR  ECM  FF,  CAS  TR  TGT  FA,  CAS  TR  CLT  FA,  and  CAS  TR  ECM  FA.  Make  certain 
that  a  appears  in  each  box.  Checking  these  boxes  informs  MAES  that  multiple  CAS 
Track  Mures  exist.  Once  the  check  boxes  have  been  selected,  press  the  “Continue”  button 
on  the  Cahbration  DSOT  Printout  Menu. 

An  ordered  list  of  paths,  in  this  case  only  one  that  says  “All  CAS  Track  Modes”,  that  MAES 
will  take  you  through  will  be  displayed.  Click  on  the  “Continue”  button  to  start  performing 
troubleshooting  tasks. 
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Figure  4-4.  Power  Hi  Task  Screen 


Perform  Tasks 

The  next  screen  is  shown  in  Figure  4-4.  MAES  asks  if  a  power  HI  condition  exists  for  all 
track  modes,  so  click  on  the  “Yes”  button  in  the  action  area.  This  action  causes  MAES  to 
display  another  task  screen  as  shown  in  Figure  4-5. 
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Figure  4-5.  Measure  TTL  Levels 


Completing  a  Task 

The  screen  shown  in  Figure  4-5  instructs  you  to  measure  TTL  Remote  Mode  Logic  Levels 
at  UD412/A1A5-A13.  If  you  require  help  on  how  (or  why)  to  perform  this  procedure,  you 
can  request  on-line  help  as  explained  in  the  next  section. 

Getting  Help 

Press  the  button  labeled  “How”  as  shown  in  Figure  4-5.  MAES  will  give  you  detailed 
instructions  on  how  to  measure  TTL  Remote  Mode  Logic  Levels  at  UD412/A1 A5-A13.  A 
view  of  this  information  is  shown  in  Figure  4-6.  Use  the  scroll  bars  to  read  the  entire  set  of 
instructions. 
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AJI  CAS  Track  FF  and  FA.  :- 

s  Power  Higfc  Cicwrs: 


m  1  Plac9  DSOTtests«t<yD412)  in  REMOTE  by  perfroming  the  following: 

I  ^  AtU0461,pre$eT0TE  SELECTN  MISCfTEST,  MINUS  UEFTCRQP 

I  DOWN,  03  and  INSERT  pu&hbuttona. 

I  t»}AtUD4T2,V«rifyREMOTEtampiStit 

P 2.  WaftuntilcalibrationtscotniJl^ertt beforetaking anymeasurements 

^  (approximate^  ^  minutea). 

1 

i  3.  AtUD412iA1AS-A13  cjieck  logic  levels  at  TP15,  TP17,  andTP22 
i  tiaiPg  a  mullnneter. 

I 

g  4.  Record  results. 

p  5.  Return  to  MANUAL  mode  wdien  test  is  completed. 


Figure  4-6.  HelpScreen  (How  to  Measure  TTL  Remote  Mode  Logic  Levels) 


Once  you  have  finished  reading  the  help  information,  click  on  the  button  labeled  “Return”. 
MAES  will  return  you  to  the  previous  screen  (Figure  4-5).  Now  click  on  the  button  labeled 
“Why”.  MAES  gives  you  a  concise  reason  why  you  are  instructed  to  perform  this 
measurement.  The  Why  screen  is  depicted  in  Figure  4-7.  After  reading  the  steps  in  the  Why 


>0<5ASTfacfc  tod«s  fFanOFA. 


display,  exit  back  to  the  Measure  TTL  Levels  by  clicking  on  the  “Return”  button. 

Since  an  all  CAS  Track  Failure  in  the  example  indicates  that  a  logic  level  at  TP  15,  TP  17,  or 
TP22  is  high,  press  the  button  labeled  “Yes”  in  Figure  4-5.  This  action  will  cause  MAES  to 


Figure  4-7.  Replace  Parts  Screen 
display  the  Replace  Parts  Screen  as  shown  in  Figure  4-7. 

Replace  Failed  Part 

When  replacing  parts  you  may  access  part  information  that  would  otherwise  take  you  hours 
to  access  via  normal  supply  channels  and  technical  manuals.  To  use  this  information,  click 
on  the  button  labeled  “Parts  Info”.  The  type  of  information  that  is  available  is  displayed  in 
Figure  4-8. 


Figure  4-8.  Parts  Information  Screen 


Exiting  MAES 

After  you  have  obtained  the  part  information  you  need  to  replace  a  part,  click  on  the  “Return” 
button.  You  are  now  back  on  the  Replace  Parts  Screen.  Clicking  on  the  button  labeled 
“Continue”  will  return  you  to  the  Calibration  Main  Menu.  You  may  now  exit  MAES.  This 
concludes  the  sample  session. 
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Documentation  and  Technical  Support _ 

Technical  support  regarding  MAES  may  be  received  from  representatives  at  NAVSEA  by 
mail^  telephone,  or  by  GENADMIN  message. 

NAVSEA  Assistance 

Hardware  and  software  technical  support  may  be  obtained  from  the  NAVSEA  technical 
representatives  at  the  Naval  Surfece  Warfere  Center,  Port  Hueneme  Division.  The  NAVSEA 
point  of  contact  is  Mr.  Henry  Seto,  and  he  can  be  reached  at: 

Mr.  Henry  Seto  (Code  4C46) 

Port  Hueneme  Division,  Naval  Surface  Warfare  Center 
4363  Missile  Way 
Bldg.  1211 

Port  Hueneme,  CA  93043-4307 
(805)  982-0141  commercial  or 

551-0141  DSN,  e-mail:  SETO_HENRY@OM.NSWSES.NAVY.MIL 
Message  Address:  RUWFPBC/NAVSURFWARCENDIV  PORT  HUENEME  CA//4C00// 

Documenting  Problems  and  Suggestions 

We  believe  that  one  of  the  reasons  MAES  is  so  effective  at  solving  faults  in  the  MK  92  PCS 
is  that  you,  the  fleet  technician,  have  been  a  key  component  in  the  development  of  this 
software.  It  was  people  like  you  viio  suggested  that  we  include  photos  and  parts  information 
which  have  become  key  features  of  MAES.  If  you  have  a  suggestion  that  you  feel  would 
make  MAES  a  more  effective  tool,  please  fill  out  the  Digital  Systems  Feedback  Report 
included  as  the  last  page  of  this  manual.  You  may  also  use  the  same  form  to  report  a 
recurrent  problem  that  may  be  occurring  with  your  software  or  computer. 
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APPENDIX  D.  MAES  USER  SURVEY 

1 .  What  is  your  current  rank? 

□  El  □  E2  □  E3  □  E4  □  E5  □  E6  □  E7-E9 

2.  How  many  years  of  experience  have  you  had  as  a  MK92  tech? 

□  Less  than  1  year 

□  1-2  years 

□  3-4  years 

□  5-6  years  . 

□  7-8  years 

□  9  or  more  years 

3.  On  what  ship  are  you  currently  stationed  on?  _ 

4.  What  is  your  current  billet? 

□  Technician 

□  Work  Center  Supervisor 

□  Divisional  LPO 

□  Divisional  CPO 

□  Other 

5.  When  you  had  a  feult  to  troubleshoot,  what  role  did  MAES  play? 

□  Did  not  use  MAES  to  troubleshoot 

□  Used  MAES  in  conjunction  with  the  technical  manuals  □  Used  MAES  only 

6.  Have  you  used  MAES  for  (check  all  that  apply): 

□  Troubleshooting 

□  Training 

7.  How  often  do  you  use  MAES? 

□  Never 

□  For  all  maintenance  troubleshooting  covered  by  MAES 

□  For  part  of  maintenance  troubleshooting  covered  by  MAES 
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8.  How  has  MAES,  in  comparison  to  only  using  technical  manuals  for  troubleshooting, 
changed  the  time  required  for  you  to  repair  a  feult? 

□  Could  not  have  solved  some  faults  without  MAES 

□  Significantly  less  amount  of  time  to  repair  faults 

□  Less  amount  of  time  to  repair  faults 

□  More  time  is  required  to  repair  faults 

□  Significantly  more  time  is  required  to  repair  faults 

Please  rate  MAES  in  the  following  areas: 


7.  Ease  of  use 

Veiy 

Dissatisfied 

1  2  3 

4 

Very 

Satisfied 

5 

8.  Ability  of  MAES  to  solve  faults 

1  2 

3 

4 

5 

9.  MAES  User  Manual 

1  2 

3 

4 

5 

10.  Viewscreen  display  (sharpness/brightness) 

1  2 

3 

4 

5 

11.  Supply  information 

1  2 

3 

4 

5 

12.  Technical  photos 

1  2 

3 

4 

5 

13.  Explanations  of  troubleshooting  steps  (Hows) 

1  2 

3 

4 

5 

14.  Explanations  of  troubleshooting 
methods  (Whys) 

1  2 

3 

4 

5 

15.  Online  help 

1  2 

3 

4 

5 

16.  Your  confidence  in  the  system 

1  2 

3 

4 

5 

17.  Reliability  of: 

Laptop  Computer 

1  2 

3 

4 

5 

MAES  Software 

1  2 

3 

4 

5 

18.  Your  overall  satisfaction  with  MAES 

1  2 

3 

4 

5 
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Comments: 
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